This documentation is for WSO2 Enterprise Integrator version 6.2.0 . View documentation for the latest release.

All docs This doc
Skip to end of metadata
Go to start of metadata

The file inbound protocol is a multi-tenant capable alternative to the VFS transport. The file inbound protocol uses the VFS transport to process files in a specified source directory. After processing the files, it moves them to a specified location or deletes them. Note that files cannot remain in the source directory after processing or they will be processed again, so if you need to maintain these files or keep track of the files that are processed, specify the option to move them instead of deleting them after processing. 

Security

The file inbound protocol supports the FTPS protocol with Secure Sockets Layer (SSL). The configuration is identical to other protocols. The only difference is the URL prefixes and parameters.  For more information, see VFS URL parameters.

Failure tracking

To track failures in file processing that can occur when a resource becomes unavailable, the VFS transport creates and maintains a failed records file. This text file contains a list of files that failed to processed. When a failure occurs, an entry with the failed file name and timestamp is logged in the text file. When the next polling iteration occurs, the VFS transport checks each file against the failed records file, and if a file is listed as a failed record, it will skip processing and schedule a move task to move that file.

<inboundEndpoint xmlns="http://ws.apache.org/ns/synapse" 
				 name="file" sequence="request" 
				 onError="fault" 
				 protocol="file" 
				 suspend="false">
   <parameters>
      <parameter name="interval">1000</parameter>
      <parameter name="sequential">true</parameter> 
      <parameter name="coordination">true</parameter> 
      <parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
      <parameter name="transport.vfs.MoveAfterProcess">file:///home/user/test/out</parameter>
      <parameter name="transport.vfs.FileURI">file:///home/user/test/in</parameter>
      <parameter name="transport.vfs.MoveAfterFailure">file:///home/user/test/failed</parameter>
      <parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
      <parameter name="transport.vfs.ContentType">text/plain</parameter>
      <parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
   </parameters>
</inboundEndpoint>

VFS service-level parameters

The VFS transport does not have any global parameters to be configured. Rather, it has a set of service-level parameters that must be specified for each proxy service that uses the VFS transport. For information on how to configure the file inbound protocol for FTP, SFTP and FILE connections, see Configuring File Inbound Protocol for FTP, SFTP and FILE Connections.

Parameter

Description

Required

Possible Values

Default Value

intervalThe time duration in milliseconds between two file scans that checks for updates.Yes

sequentialFiles will be processed sequentially when this parameter is set to true.Yes
true
coordination

This should be true for clustered deployments in order to prevent two nodes from retrieving the same file.

Yes
true

transport.vfs. FileURI

The URI of the location of your files. This should be the source location of the files (if you are configuring the ESB to read files) or the destination of the files (if you are configuring the ESB to send files). You can specify connection-level parameters on the URL (see VFS URL parameters).

When you need to access the absolute path of the URL, you can define the URL with sftpPathFromRoot as shown below. Also, note that transport.vfs.AvoidPermissionCheck is a mandatory parameter for this URL when FTPS is used.

<parameter name="transport.vfs.FileURI">sftp://[ username[: password]@] hostname[: port][ absolute-path]?sftpPathFromRoot=true;transport.vfs.AvoidPermissionCheck=true</parameter>

Yes

A valid file URI in the following form:
file://<path>


transport.vfs. ContentType

Content type of the files processed by the transport. To specify the encoding when reading a file, follow the content type with a semi-colon and the character set. For example:

<parameter name="transport.vfs.ContentType“>text/plain;charset=UTF-32</parameter>
When writing a file, you can set a different encoding with the CHARACTER_SET_ENCODING property:
<property name="CHARACTER_SET_ENCODING" value="UTF-8" scope="axis2" type="STRING"/>

Yes

A valid content type for the files (e.g., text/xml). You can specify the encoding after the content type, such as: text/plain;charset=UTF-32


transport.vfs. FileNamePattern

If the VFS listener should process only a subset of the files available at the specified file URI location, use this parameter to select those files by name using a regular expression.

No

A regular expression to select files by name (e.g., *\.xml)


transport.vfs. ActionAfterProcess

Whether to move or delete the files after the transport has processed them.

No

MOVE or DELETE

DELETE

transport.vfs. ActionAfterFailure

Whether to move or delete the files if a failure occurs.

No

MOVE or DELETE

DELETE

transport.vfs. MoveAfterProcess

If ESB reads the files without any issue, it moves the files to the location specified in MoveAfterProcess parameter.

Yes, if
ActionAfterProcess
is MOVE

A valid file URI


transport.vfs. MoveAfterFailure

If ESB is unable to read a file from the location specified in FileURI due to an error while reading the file, it moves this file to the location specified in MOVEAfterFailure .

Yes, if
ActionAfterFailure
is MOVE

A valid file URI


transport.vfs. ReplyFileURI

The location where reply files should be written by the transport.

No

A valid file URI


transport.vfs. ReplyFileName

The name for reply files written by the transport.

No

A valid file name

response.xml

transport.vfs. MoveTimestampFormat

The pattern/format of the timestamp added to file names as prefixes when moving files.

No

A valid timestamp pattern
(e.g., yyyy-MM-dd'T'HH:mm:ss.SSSZ )


transport.vfs. Streaming

Whether files should be transferred in streaming mode, which is useful when transferring large files.

No

true or false

false

transport.vfs. ReconnectTimeout

Reconnect timeout value in seconds to be used in case of an error when transferring files.

No

A positive integer

30 sec

transport.vfs. MaxRetryCount

Maximum number of retry attempts in case of errors.

No

A positive integer

3

transport.vfs. MoveAfterFailedMove

The location to move a failed file.

No

A valid file URI


transport.vfs. FailedRecordsFileName

The name of the file that maintains the list of failed files.

No

A valid file name

vfs-move-failed-records.
properties

transport.vfs. FailedRecordsFile Destination

The location to store the failed records file.

No

A folder URI

repository/conf/

transport.vfs. MoveFailedRecord TimestampFormat

The time stamp format for entries in the failed records file. The failed records file maintains the name of the file that failed and the timestamp of its failure.

No

A valid timestamp pattern
(e.g., yyyy-MM-dd'T'HH:mm:ss.SSSZ )

dd-MM-yyyy HH:mm:ss

transport.vfs. FailedRecordNext RetryDuration

The time in milliseconds to wait before retrying the move task.

No

A positive integer

3000 milliseconds

transport.vfs.Locking

By default, file locking is enabled in the VFS transport. This parameter lets you configure the locking behavior on a per service basis. You can also disable locking globally by specifying the parameter at the receiver level and selectively enabling locking only for a set of services.

No

enable or disable

enable

transport.vfs. FileProcessCountThis parameter allows you to throttle the VFS listener by processing files in batches. Specify the number of files you want to process in each batch. If you specify a value for thisNoA positive integer, such as 10N/A

transport.vfs. FileProcessInterval

The interval in milliseconds between file processing batches.NoA positive integer, such as 1000N/A

transport.vfs.DistributedLock

This applies only in cluster deployments. Set to true if you need to avoid multiple servers trying to process the same file simultaneously.Notrue or falseN/A

transport.vfs.DistributedTimeout

The timeout period in seconds for the distributed lock.NoA positive integer, such as 10N/A

transport.vfs.AutoLockRelease

Set to true if you need to release locking in order to avoid files not being processed due to faulty locking. This works together with the transport.vfs.AutoLockReleaseInterval and transport.vfs.LockReleaseSameNode parameters. Notrue or falseN/A

transport.vfs.AutoLockReleaseInterval

Lock release interval in milliseconds.NoA positive integer, such as 1000N/A

transport.vfs.LockReleaseSameNode

Set to true if you need to release the locks only accrued by the same worker node.
If this is set to false, locks accrued by other nodes will be released according to the value specified in transport.vfs.AutoLockReleaseInterval .
Notrue or falsetrue
transport.vfs.FileSortAttributeThe attribute by which the files should be sorted and processed.NoNONE, Name, Size and LastmodifiedtimestampN/A
transport.vfs.FileSortAsscendingThe sort order to sort and process the files. If set to true files will be sorted in ascending order based on the attribute you specify in transport.vfs.FileSortAttribute. Notrue or falsetrue
transport.vfs.CreateFolderSet to true to create a folder if a folder does not exist when moving files.Notrue or falsefalse
transport.vfs.SubFolderTimestampFormatThe pattern/format of the timestamps added to the folder structure when moving files. You need to set transport.vfs.CreateFolder to true to in order to specify a value for this parameter.NoA valid timestamp pattern 
(e.g., yyyy-MM-dd'T'HH:mm:ss.SSSZ )
N/A
transport.vfs.BuildSet to true if you need to build the content inside the file before injecting the file to the mediation engine. If there is a build error, the file will not be injected to the mediation engine.Notrue or falsefalse

Note

If you specify the transport.vfs.FileProcessCount parameter you do not need to specify the transport.vfs.FileProcessInterval parameter in a configuration, and vice versa. This is because the transport.vfs.FileProcessCount parameter and the transport.vfs.FileProcessInterval parameter cannot be used at the same time.

Samples

For a sample that demonstrates how to use the file system as an input medium via the inbound file listener, see  Sample 900: Inbound Endpoint File Protocol Sample (VFS).

  • No labels