Appreciating BAI Files [Part 3]

A Primer for Bank Statement Automation Part 3: Implementation Considerations

The requirements of a BAI implementation are simple: deliver a file from the bank to its target customer. That’s it. However; BAI implementations tend to come across as complex projects that many times take several weeks to complete. I attribute two fundamental reasons for the unnecessary complication of this process: (1) non-technical parties get placed as the decision-makers for technical details governing the mechanical components of the process; and (2) technical people have a strong bias as to what works and does not work within their technical ecosystem. The definitive resolution to either of these issues is fundamentally impossible, but perhaps the following considerations can help narrow the (communication) gap and expedite the decision-making process.

FTP vs. FTPS vs. SFTP…. and SFTP+PGP?

The extent of the technical jargon over BAI implementations is tremendous, and the maturing of newer file exchange protocols (ISO 20022 XML – I’m looking at you) is only adding to the terminology and options available for information exchange. For practical purposes, I will focus on the tried and true flavors for BAI file connectivity and transmission. Essentially, there are two “universal” options for enabling this file exchange: SFTP and FTPS (FTP alone is not secure and we will get to PGP later). From a functional perspective, both options are essentially the same – they securely move a file from one domain to another. From a technical perspective, additional details make this more a matter of opinion or corporate preference. Below is a breakdown of the moving pieces.

FPT = File Transfer Protocol

• The de-facto standard for exchanging files over the Internet.

• Standard was not designed to be a secure protocol (i.e. all transmissions are in clear text).

• The protocol has been extended to serve in secure transmissions (see variations below)..

FTPS = FTP with SSL (Secure Socket Layer) certificate.

• Requires a digital certificate to validate connection and transfer data.

• It is widely known and used, and provides a variety of services for server-to-server file transfer.

• Can be tricky to configure behind firewalls as it requires a two channels (data and control).

SFTP+PGP

• PGP (Pretty Good Privacy – I kid you not) is an encryption mechanism so files are unreadable if intercepted.

• Good idea if SFTP only relies on username /password for authentication (vs. SSH keys), otherwise is overkill.

• Does not apply to FTPS thanks to the certificate.

• One could make a good case for standard FTP with PGP as information is secured by encryption.

My personal opinion? I would go with SFTP and exchange SSH keys with the bank. I find this to be a faster implementation (particularly if the client does not already have an SSL certificate) as it lends itself for more work to be completed without the direct coordination of a second party. A big factor to consider is support and maintenance post-implementation. If this is an issue, FSTP is arguably the way to go. I would contend, however; that the correct answer to protocol selection is the one offering the path of least resistance. In some circumstances, adhering to the preferred standard at the bank may lead to a faster implementation. Maybe not. Either way, don’t lose perspective that the purpose of the implementation is to pass a simple text file from point A to point B.

Pull vs. Push

Now that the core mechanical components of the implementation have been described, there is one more decision to consider: Push or Pull. Push and Pull refer to the file delivery mechanism over the FTP interface. In simple terms, Push means that the publisher of a file is submitting or “pushing” the file to its intended recipient, while Pull implies that the intended recipient is initiating the request to obtain a file or “pulling” the file from its publisher. In the world of BAI, the most common implementation is pull where the customer will initiate the FTP request to pull data files from a directory on the server. Typically, the bank will assign an outbound directory within a server to the customer and place a BAI file in that target directory whenever it is available. In turn, the customer will initiate a “pull” script to retrieve the file from the outbound directory to their domain or local system.

Okay… pull/push…. who cares?

Fair question since both require a similar level of coordination by all parties. The key distinction that may serve as a deciding factor is the overall intended use of the interface. If the interface established at the bank is only going to facilitate the transmission of BAI files, pull probably makes the most sense. If, however; the interface is going to also include the exchange of other information such as Positive Pay or wire /ACH transactions, push may be the better candidate. With push, the publishing party has greater control over when information is available to the receiving party. This is important for time-sensitive files such as wires. This may also be a factor if the interface is intended for Current Day BAI files, but banks tend to like to treat all BAI files as a pull interface. Some banks will allow a customer to leverage a single interface to facilitate all transactions types (i.e. BAI, positive pay and wire/ach) separated by different directories. Other banks may require customers to build separate interfaces for each file type. When designing a content management system for the bank interface, consider all files that will /may participate in the process and to consolidate as many of these within the same interface to reduce maintenance.

Directory Structures

Files need a home - below are considerations regarding directory structures.

Even though connectivity and transmission may be neatly out of the way, the BAI implementation is not complete without considering how these files will reside after they are received (or collected) from the bank. This is particularly important if the implementation involves more than one bank. Having a sound directory structure is critical to a successful process and that may require some planning. For most implementations, I like having a Staging directory and an Archive folder for each bank to keep things organized. The Staging directory is the target folder for the application consuming the BAI file. The Archive folder serves as a place to store files that have already been processed as history. Most modern FTP programs come with automation features to facilitate the movement of files, but many times I still rely on simple batch scripts (Windows) to scan folders and move files to different directories to accommodate special circumstances. One common issue affecting directory structures is that many banks tend to reuse the exact same filename for all BAI files. This creates an issue for archiving but is easily circumvented by applying a date/time to the filename. This scenario does become dangerous, however; if the target directory for the interface is not cleaned out before a new file is published. Another issue is that of data security, i.e. encryption at rest. Assuming files are encrypted, a simple solution is decrypt the target file into the Staging directory and delete after processing while saving the encrypted file in the Archive folder. It is easy to write-off directory structures as a trivial part of the implementation. Don’t be caught unprepared. The bank may also help fix several of these issues if they are identified and discussed as the implementation is taking place (many times these items are identified as problems after the fact).

Scheduling Considerations

One last consideration regarding a BAI implementation is scheduling, i.e. finding the appropriate timing to run the interface and consume the data. Prior Day files are typically available by (or before) 7am. Current Day files are available throughout the day in “windows” but are mostly driven by business-driven events, such as posting of controlled disbursements. Unfortunately, there seems to be more success judging the scheduling of interfaces based on historical results than by following time tables provided by the bank. For most implementations, I like to schedule an interface to run at least 20 minutes later than the “publishing” time. I also like to schedule 3 attempts that are 5 minutes apart for receiving the file before declaring a failed process (this is when the BAI department at the bank should be contacted for a status – BAI files presented late is not uncommon). I also like running the archiving process at the end of the day to ensure that no files are present in Staging directories before the next cycle begins. Following these practical considerations may help prevent some maintenance headaches when the interface goes into Production.

If you want to know more about BAI files, below are links to the entire series:

Part 1: Meet the Standard

Part 2: Breaking Down the BAI File Structure.

Part 3: Implementation Considerations

Part 4: Are BAI Files Perfect?