Web application configuration on Tomcat using Spring

In order to get two sets of independent configuration (property) files, one for development and one for production, one can do the following:

In the web-inf folder place a property file, e.g. act.properties
In the applicationContext.xml define:




WEB-INF/act.properties
file:///${CATALINA_HOME}/act.properties

Birt versus Jasper Reports

My experiences as a new user with Birt and Jasper.

Birt

• A report grouped by week done within a one - two hours
• Built-in support for group-by week (special function)
• Adding JDBC drivers is easy

Jasper

• I get hopelessly lost in the NetBeans based IDE. I do not understand how to use the query builder and the report builder.
• Trying to build a report with group-by week is hard. No built-in support for weeks, just the standard java.util.timestamp functions. Trying to build an expression using day() results in weird validation errors (cannot cast from int to Integer()). With 1.5 you should not get this error anyway. Adding cast does not help. Bottom-line I do not understand the problem and the tool is not helping me to solve it.
• NetBeans based IDE looks a lot sharper as the 3.0 classic IDE.
• Selecting a SQL Server JDBC driver from dropdown result in classnotfound exception, you can select the driver, but it is not delivered with the product. You need to add your own drivers. Very confusing
• A report needs to get compiled. Makes it harder to work with. Probably results in better performance but should be optional.
  

VB6 & REST

I want to call a REST service from my VB6 Macro. This page describes how to do this using IE 6 COM objects, instead of needing to install the MS SOAP Toolkit (if it is still available)

It will probably also work for IE 7 since the difference between the two is small. Anyway something to verify.


In case the original page gets lost:

' Create objects to DOMDocument and XMLHTTP
Set objDom = CreateObject("MSXML2.DOMDocument")
Set objXmlHttp = CreateObject("MSXML2.XMLHTTP")

' Load XML
objDom.async = False
objDom.loadXML XmlBody

objXmlHttp.open "POST", AsmxUrl, False

objXmlHttp.setRequestHeader "Content-Type", "text/xml; charset=utf-8"
objXmlHttp.setRequestHeader "SOAPAction", SoapActionUrl

objXmlHttp.send objDom.xml

strRet = objXmlHttp.responseText

Set objXmlHttp = Nothing

' strRet contains result

Evaluation criteria Java Library

Evaluation criteria I suggest:
- maturity of the product; years in service,
- size of the community, popularity among developers, documentation
- feature set and what is missing
- how easy to extend
- rate of new releases
- stability, performance, scalability
- integration with other 3rd party libs (e.g. Spring, Hibernate, ActiveMQ, GWT, etc)
- (commercial) support
- license
- Readability / ease of use of the configuration files

Partitioning

Partitioning is a very import aspect of software development; you need to partition the application into several parts; modules, classes, units, whatsoever. Dividing the work among people is also partitioning. Choosing the right partitioning is about trade-offs.

It stroke me during my vacation that a landmass like Europe is also partitioned; it is divided into countries. It took a couple of thousand of years and a lot of fights, but finally we have a stable partitioning of land. In a way it is a self organizing system.

RBAC versus ACL

RBAC stands for role-based access control
ACL stands for access control list

RBAC adds an indirection level - role.

Perfection

Perfection kills innovation

Hibernate and BI

Business Intelligence (BI) requires a star schema for efficient analytics. (the difference between analytics and reporting is that reporting requires you to know up front what you want to report and analytics allows you to do ad-hoc reporting). BI operates in most cases from a data warehouse based on star schemas to allow fast and time-based (historical) reporting. Reporting from the operational data sources would jeopardize the transactional/operational systems and in most cases does not allow for historical reports.

An ordinary application based on a persistence engine like Hibernate uses obviously an operational schema - making analytics hard. It would be nice to let Hibernate also update a data warehouse - similar to updating the indexes for text based retrieval.

So when a transaction is comitted, a message (asynchronously) is send to a component that updates the data warehouse. Each operation has a time stamp - allowing time based reporting. You could even use the data warehouse as a kind of trash bucket to restore data that has been deleted inadvertently.

To summarize; the application has three data sources; operational, warehouse and text-indexes.
An application in this way allows for text based search and analytics reports.

Tomcat as Windows Service

To install Tomcat5 as a service follow the "Windows service HOW-TO" instructions on the Apache Tomcat side.

Calling Tomcat5 //IS//MyServiceName blabla will create an entry in the Windows registry located at:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\MyServiceName

The new installed service when started does a call back to tomcat5.exe specifying the service name and the start parameter.

With tomcat5w //ES//MyServiceName you can monitor the service. Bit similar to the Windows Services application, but it has more functionality like specifying the log level and log location.

In my case the service did not start and I had an error in the log:

[2008-03-05 15:55:24] [443 javajni.c] [error] FindClass org/apache/catalina/startup/Bootstrap failed
[2008-03-05 15:55:24] [997 prunsrv.c] [error] Failed loading main org/apache/catalina/startup/Bootstrap class
[2008-03-05 15:55:24] [1260 prunsrv.c] [error] ServiceStart returned 3

So a classpath problem. I re-read the instructions and I noticed that the remark " If using tomcat5.exe, " put me on the wrong foot. I thought service.bat applied to versions prior 5. Appearently this is not the case because running service.bat worked fine. Looking into the content of service.bat I noticed that there is some classpath manipulation and after installing the service, the service is also updated with JVM options so this way is superior as to the excerpt installing Tomcat5. (In hindsight I completely misinterpreted this)


Deleting the service using //DS removes the entry from the registry.

GWT & object serialization

When writing GWT business applications you have the problem that the object graph resided at the server has to be partially send to the browser/client, (perhaps) updated and send back.

A though problem to crack is how to break the graph in pieces. After all, we do not want to send the entire database from the server to the client. This is similar to the problem that Hibernate solves when communicating with the database. Hibernate will load on-demand needed objects from the database. Hibernate will also optimize the updates to the database.

I think it is key to have something similar to Hibernate for the GWT arena. Instead of manually loading additional objects a framework should handle this.

A key difference is that (in most cases) the Hibernate instance is a singleton, thus allowing caching techniques. Caching at the browser is tricky since other clients can have updated the data.

My hope was that a framework like XSTM would solve this problem. The sad things it that it uses code generation. I'm not in favor of code generation and especially not when it involves the domain objects. Reason is that it makes the usage of the library very intrusive through your entire application. Advantage of code generation in this case instead of byte code manipulation is of course that it runs in GWT.

Anyway when I look at the sample the objects replicated are stored in a share, which is a generic container and thus requiring a lot of casting. My idea that you could retrieve objects with a query kind of language is not the case, you have to manually put in the share and get it out. So this solution will work well for games I suspect - but not for business applications with huge databases. In this case you have to write code at client side to notify the server which objects to put in the share. Again plumming that I hoped the framework would fix. Looks like that XSTM requires an additional layer of abstraction on top of it that will make it hide the clouds/shares and just work with regular objects.

All problems in computer science can be solved by another level of indirection

Finally found the guy(s) whom the quote is from.

Butler Lampson and/or David Wheeler

Cross-region software development

At my company we have been struggling finding out the optimal partitioning of the source tree of a product that is developed in three different countries (US, India and NL).

Some people prefer to have a single source tree. Others (among myself) prefer to have a stricter separation between the teams in different locations and time zones. One of the major reasons is that if the build is broken or disfunctional caused by team A, it impacts the productivity of team B and C. The developers of team B or C enter in the morning and face a build in which they cannot check. First the failure has to be solved before continuing regular development. Admitted, this can be done by a single person, but still the broken build impacts all the developers of a team since they can't check-in and might have updated from source control before realizing something was wrong. Since a team performs optimal when it is running at constant speed - similar to a car's engine - you need to avoid peaks and breaks.

Another reason for keeping sources separate is that you want to avoid mixing integration and regular development cross teams. Otherwise you cannot isolate external factors easily. It is much more efficient when you have accomplished your work of team development, all tests are passing and then integrate new releases of other teams or 3'rd party artifacts. If something fails, the cause is much easier to spot then when you also have changed your code.

Lesson: separate daily development from integration. We effectuated this (or tried to) by giving each team and the application its own Maven artifact. Admitted, you need Maven mavens to get the whole thing working. (BTW we never got to the point where we could fully demonstrate this Maven approach to be more efficient, in the past we used branches)

An often heard counter argument for having separate source trees is the price you have to pay for refactorings, especially renaming classes / methods. This can be solved in the following way: at the end of an iteration, the teams have released their artifacts and hopefully the product is working with all the parts integrated. So the start of the next iteration is an excellent time to get all the source code into a single view / project and refactor some elements. Yes, indeed you do have to wait. That is a small price to pay. However note that for really big products, getting all the source code into a single project probably makes the IDE too slow. This is something you do not want to work with the whole day. Besides renaming there is also a refactoring like changing the signature. The same can be accomplished with deprecating and introducing new methods.

Another mistake often made is the focus on compilation - which can easily be accomplished with a single source tree. IF the code compiles together then we are OK. But compiling is only just a small part of creating the product, much more important is to have a stable product and you know what is in it. That the code compiles together, does not mean a thing. It gets much better if the artifacts have passed their unit tests, are released and composed into a new product - that should pass some automated acceptance test.

There is much more to this discussion, I will have to add that another time.

Panta rhei

Everything flows - Heraklitus.

Well known fact; when you write down the specs, the world is changing. By the time you've finished the implementation the world is no longer the same. This why we all went agile.

There is another side to this; with an agile approach you take shortcuts in the first iteration. Most simple thing you can do, blabla bla. However a couple of iterations down the road, you forgot you took some shortcuts. People start cursing the whacky implementation, forgetting that this was a deliberate choice and now the time has come to crank up the implementations.

Iterations also pose a challenge for QA. In my experience they are still used to the waterfall process. It is hard for them; each iteration new functionality is delivered but still some things are not (yet) fully functional. The deliverable of each iteration has to be carefully defined to enable them to properly test the deliverable. However, how much time you are going to invest writing a detailed document that is obsolete after the next iteration? Worse, how are you going to get the full spec after all iteration are finished? How efficient are UI designers if they know that each iteration the UI will change? Do they start with a wizard or tab-based approach because they know the next iteration requires more widgets?

There is some kind of impedance mismatch between the development process (iterative) and the product as itself. As usual a trade-off.

Scala

I've been reading the “Scala for Java Refugees” series. I'm amazed by the power of this language. Tuples, closure, etc, everything that annoys me about Java is 'fixed'. The language itself is a bit overwhelming, reminds me of university with its magical and impressive calculus notations.

Reading on the implicit type conversion in serie 6, I got similar feeling when reading on AOP for the first time. It might become complex to understand the code, especially in the case of Scala where a smart compiler can find paths the developer has not thought off.

Probably the answer is the same as with AOP, use it scarce, wise and effective. In other words not all over the place, but where it pays off, eg security, transactions, all those clear cross cutting concerns.

I'm impressed by the expressive power of the language. On ScalaTest the expect construct is close to natural language (ok, a bit exaggerated). The freedom it gives to clearly state the intent, is something you never get with Java.

JODReports and MS-Word

I finally implemented my document generation application using JODReports. The basic idea is as follows; user creates a bunch of templates using the JODReports variable mechanism and stores them in a certain folder. My application picks the templates up, converts to OO format, merges it with the data from the database, converts it back to Word and writes the files to a certain folder.

Works pretty good so far. A couple of issues I stumbled into:

• Editing the Word document by pasting a variable name from html (in my case the instructions page in the browser) gives weird Freemarker parse exceptions. Pasting as plain text workarounds the problem.
  
• In the supplied merge data the zip code field is separated by two spaces from the city name. However the final merged document contains one space. This is also the case if the data contains three spaces...
  
• Default error handling from Freemarker prints out to the console. There is documentation how to fix this, but requires more research.
• JODReports code is not well maintained. The code of the converter and the report are out of sync (watch the package names). Luckily this does not pose a problem when using it.
  

It is a bit of a pity that JODReports does not support callbacks. Now you have to pass in all data up front. It would be nicer if JODReports would do a callback asking for the value of a field. Reason is that I need to supply a couple of calculated fields. Now I have to supply them all up front, whilst perhaps none is used.

Java Bug Date equals?

Who would have thought that the following test would fail:

private Date getDate14April() {
Calendar calendar = Calendar.getInstance();
calendar.set(2007, 3, 14, 15, 30); // 14 april 15:30
return calendar.getTime();
}

public void testCompareDate() {
Date date = getDate14April();
for (int i=0; i < 100; i++) {
assertEquals(getDate14April, date);
}

Stacktrace (edited brackets):

junit.framework.AssertionFailedError: 60 expected: [Sat Apr 14 15:30:40 CEST 2007] but was:[Sat Apr 14 15:30:40 CEST 2007]
at org.westen.act.document.TestTemplateManager.testCompareDate(TestTemplateManager.java:212)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)


JavaDoc of Date.equals:
Compares two dates for equality.
The result is true if and only if the argument is not null> and is a Date object that represents the same point in time, to the millisecond, as this object.

Hmm, I would have though that the milliseconds would have been initialized to 0 when I create a date object.

Hibernate many-to-many & inverse

I had the problem that two entities A and B had a many-to-many relation but deleting an object a resulted in a constraint violation because the object was used by an object b.

To my surprise fiddling with the inverse="true", inverse="false" and cascade="none" fixed the problem.

My understanding is that deleting an object always deletes the row from the join table. (cascade="none"). Setting cascade to "delete" will also force the deletion of the object(s) on the other side.

However why inverse="true" and "false" make a difference I do not understand. You would assume that specifying one would be enough and imply the other. Apparently there is more to it than I understand. Needs a follow up...

So I'm not the first one to run into this. Found this blog "Hibernates bizarre interpretation of inverse". He basically says that inverse tells Hibernate which side of the relation to ignore. Although this does make complete sense to me, it fits my problem.

Hmmmm

GWT-EXT

The last few weeks i've been struggling with GWT-Ext 1. Conclusion: the lay-out capabilities of GWT-Ext 1 are poor.

For example a table-layout is not possible. When using a form you can only place a limited number of widgets on it. Separate label widget is not present.

Ext v2 has much richer lay-our support. So we have to wait for GWT-Ext 2....