UltraESB File Handling Sample

It is a problem that someone have to continuously monitor a folder or group of folders for new files then trigger a custom action such as categorizing them, convert into desired format or response them etc

So this article demonstrates the capability of the Free and  Open Source UltraESB to pick up a file from a directory, process it within the ESB and finally move the files to corresponding directories after processing, depending on the outcome.

If you installed the UltraESB v1.4.0 runtime version you’ll find all available examples inside the “ultraesb-1.4.x/samples/conf ” directory

( /home/imalka/java/ultraesb-1.4.0/samples/conf  with respect to mine)

Today I am going to work with the sample-401 (ultra-sample-401.xml configuration file) . To try out this sample you should first create the directory structure to store the files that need to be processed and the files after processing

According to the configuration, my directory hierarchy is created in the “/tmp” directory in linux operating system. The top level directory is named as “file”. One can create this folder structure in any location on the computer where you have permissions to update (instead of /tmp directory) 

Inside “/tmp/file” directory

incoming - the directory which contains all the files which needs to be processed

To store processed files,

error - contains failed files after processing

done - contains successfully processed files

sent  - we are not discuss about this directory in this article. But you can find an  example related to this in sample-401(in file-proxy5)

Running the sample

Executing configuration

Then start the UltraESB configuration, execute sample 401 as follows.

                 

In my case,
imalka@imalka:~/java/ultraesb-1.4.0/bin$ ./ultraesb.sh -sample 401                     

Insert files into “ /tmp/file/incoming ” diectory

Now you can create files according to the configurations define in sample 401.

file-proxy1

<u:proxy id="file-proxy1">

        <u:transport id="file">

            <u:property name="url" value="file:///tmp/file/incoming"/>

            <u:property name="fileNamePattern" value=".*\.txt"/>

            <u:property name="startDelay" value="1000"/>

            <u:property name="repeatInterval" value="2000"/>

            <u:property name="moveAfterProcess" value="/tmp/file/done"/>

            <u:property name="moveAfterFailure" value="/tmp/file/error"/>

            <u:property name= "moveTimestampFormat" value="yyyy_MM_dd_'T'HH_mm_ss.SSSSZ"/>

        </u:transport>

        <u:target>

            <u:inSequence>

                <u:java><![CDATA[

                    String fileName = msg.getFirstTransportHeader("FileName");

                    System.out.println("Got file : " + fileName);

                    if ("error.txt".equals(fileName)) {

                        throw new Exception("This file should fail");

                    }

                ]]></u:java>

            </u:inSequence>

        </u:target>

</u:proxy>

The sample below shows the configuration used in the example, which can be easily modified and extended. The example configuration we use is as follows. it uses the method msg.getFirstTransportHeader() API by passing corresponding file name as a String.

This code snippet in the configuration (UltraESB sample # 401) defines a proxy service that start up after 1 second and polls for "*.txt" files at /tmp/file/incoming directory every 2 seconds. Processed and failed files are moved to /tmp/file/done or /tmp/file/error directories with a time stamp.  This proxy throws an exception if the filename patches 'error.txt' so that the received files maybe tested.

Creating files for testing
 

Create two files, “test.txt” & “error.txt” inside “/tmp/file/incoming ” directory. After processing those two files “test.txt” file will be moved into “/tmp/file/done” directory and “error.txt” file be classified as failed file because of its name and move into “ /tmp/file/error ” directory.

file-proxy3

To try out this you should first start a test server to simulate a web service 

The UltraESB ships a sample Jetty server with web services one could use for testing. This could be started using the ToolBox as follows,

imalka@imalka:~$ cd /home/imalka/java/ultraesb-1.4.0/bin/

imalka@imalka:~/java/ultraesb-1.4.0/bin$./toolbox.sh                                                               

Now AdroitLogic toolbox for Ultraesb window will be appear in your work place.

Go to file menu  -->  new  -->  Jetty Server (or Ctrl + j)

Start Jetty (in port 9000)

 

To get a better idea of  AdroitLogic ToolBox for the UltraESB refer the following link  

Getting started with the AdroitLogic ToolBox for the UltraESB
 

<u:proxy id="file-proxy3">

        <u:transport id="file">

            <u:property name="url" value="file:///tmp/file/incoming/response.test"/>

            <u:property name="startDelay" value="1000"/>

            <u:property name="repeatInterval" value="2000"/>

            <u:property name="moveAfterProcess" value="/tmp/file/done"/>

            <u:property name="moveAfterFailure" value="/tmp/file/error"/>

            <u:property name="moveTimestampFormat" value="yyyy_MM_dd_'T'HH_mm_ss.SSSSZ"/>

        </u:transport>

        <u:target>

            <u:inSequence>

                <u:java><![CDATA[

                    String fileName = msg.getFirstTransportHeader("FileName");

                    System.out.println("Got file : " + fileName);

                    msg.holdCompletion();

                ]]></u:java>

            </u:inSequence>

            <u:inDestination>

                <u:address>http://localhost:9000/service/EchoService</u:address>

            </u:inDestination>

            <u:outSequence>

                <u:java><![CDATA[

                    if ("response".equals(mediation.readPayloadAsString(msg))) {

                        System.out.println("Expected response received");

                    } else {

                        throw new Exception("This file should fail");

                    }

                ]]></u:java>

            </u:outSequence>

</u:target>

The above code snippet defines a proxy that polls /tmp/file/incoming/response.test every 2 seconds and holds off message completion by using the method msg.holdCompletion() API until the file payload is sent to an HTTP endpoint at http://localhost:9000/service/EchoService and a response is received. Success or failure in file processing is depend on the result of server response.

Creating files for testing

Create two files and name them as, “ response.test ” (inside “/tmp/file/incoming ” directory) and type “response” and “test”.

When processing two files poxy service searching for the expected results (the content “response”) inside the files. 

 After processing only the file which contains the expected content will be moved into “/tmp/file/done” directory and the other file will be moved into “ /tmp/file/error ” directory.

Note :

Instead of throwing exception you can use setMarkedAsFailed() method in Message class.

Just replace,

throw new Exception("This file should fail");

line in “ ultra-sample-401.xml ” cofiguration file with,

msg.setMarkedAsFailed(true);

This method mark the message as failed, by passing true. For more details on that method please refer  the Javadoc API .