Overcoming the challenges of electronic data interchange (EDI) implementation
– By Hiten Shah & Anjana S
Transportation & Logistics
Digital Transformation
In today’s global economy, reducing the time and cost required to satisfy customer requests are keys to differentiating you from your competitors. In our last white paper “EDI: The Legacy Standard Continues to Thrive in Transportation Industry” we discussed EDI and its benefits in Transportation and Logistics Industry. Implementing Electronic Data Interchange (EDI) allows companies to handle business transactions with any trading partner using a standardized electronic format. EDI enables businesses to eliminate cost and inefficiencies from transactional processes.
In spite of evident advantages of the EDI system, it can result in both positive and negative effects for businesses. This white paper discusses challenges of EDI implementation and how to overcome them.
Challenges of EDI implementation
- Technology limitations: The IT industry is continually evolving with frequent introductions of new technologies, and companies need to adapt them to remain competitive in the market. Often these new technologies are introduced with the necessity to acquire new skills, implement new infrastructure and other internal reorganization which can be a challenge, especially for small or medium-sized companies. Software maintenance, upgrades and monitoring EDI system 24/7 when issues arise play a significant role towards failure.
- Difficulty in meeting changing business requirements: Dealing with continually evolving business requirements is unavoidable. Especially those that require their IT department to reconfigure/modify/extend their internal IT systems along with their existing business partner EDI integration. Some changing business requirements include:
- Implementing a new communication technology
- Introducing new business rules which impact EDI mapping in many cases
- Add new validations, restrictions, policies in EDI which compel IT to reconfigure/modify/extend the existing EDI system
- Scaling your EDI: To scale your EDI, you need to:
- Ensure your EDI software accommodates multiple sets of trading partner business rules and document types. Business rules include individual document fields, data elements and business rule-driven validations, like store number or ship-to location
- Integrate EDI transactions into your ERP systems to improve efficiency
- Consider outsourcing your EDI program to a third party EDI provider
- Implementing in-house EDI can be quite expensive: The cost of implementing EDI depends on the approach you take - in-house or working with a third-party EDI provider. Other significant costs are related to EDI infrastructure, external mapping consultancy, EDI software license fees and on-going employee training.
- Missing details on returned documents: Trading partner may not be able to import or export all of the data that is present on the inbound document, and which is mandatory on the returned documents.
- Lack of interoperability, flexibility, and scalability: Many perceive EDI as a data format; it would be more accurate to see that EDI as a system for exchanging business documents with external entities, and integrating the data from them into the company's internal systems.
- Managing variety of integrations is difficult: Organizations may start by running a few in-house combinations. But, as the number and variety increase, this can become distracting and present scalability issues. The net effect of this on the supply chain network is sizeable and has an inevitable consequence of increasing the supply-chain costs.
-
Too many standards: To exchange documents with trading partners, you must convert the data to and from EDI Standard formats, which are controlled by a governing body. But within this standard, EDI allows industries and companies the flexibility to create their own rules, which can be modified whenever trading partners decide a business change.
Some of the formats in use - EDIFACT, ANSI, X12, TRADACOMS, XML, JSON, IMP, CSV, ODETTE, VDA, VICS, HIPAA, EANCOM, ebXML, RosettaNet, in-house formats, and national standards. Unfortunately, many EDI applications do not or easily support these conventions, which would then result in expensive upgrades. Chances that some of your partners are using a format that your systems cannot understand is high, so you need to have a solution in place that can translate the information into a data that your systems can understand which can be an overwhelming task.
- Cost or service concerns: Costs can escalate unreasonably, or you are unable to adapt the service to suit your changing business needs, which arises due to various reasons. For example:
- A variable pay-as-you-use model, example KiloCharacter charging, seemed attractive when you started EDI. At that time, the volume of documents exchanged was low. But now, since your business sends larger document volumes, it will cost you a lot more.
- You may have signed for a deal at a desirable price, and later you found upgrading cost or extending the services to be prohibitive.
Some of the problems that frequently occur even in stable EDI operations:
- Resolving unacknowledged outbound messages —Functional Acknowledgement reconciliation (997, 999, CONTROL) and FTPS message delivery notices (MDN).
- To restart halted processes.
- Correcting illegal characters —processing stoppers. Example: ' (grave accent).
- Correction or notification of incomplete documents (810, 855, 856) that erred out due to missing or incorrect data. Examples: PO # too short or Part Number.
- Repairing broken transmission (all customers) —perhaps an I.P. address changed.
- Repairing broken transmission (one customer) —perhaps a certificate expired.
- Providing proof of delivery of Invoices (810, 880, INVOIC) and Shipment Notices (ASN, 856, DELFOR) —to speed cash flow.
- Reading and correcting error messages —(864) Select messages always copied to DCS.
- By resuming AS2 firewall connections. Example: a port erroneously closed during a security scan.
- Performing maintenance routine - weekly, monthly, quarterly, and end-of-year.
- Updating non-integrated print and screen entry forms.
- Updating customer required modifications. Example, changing from ship date to arrival date.
- By monitoring B2B providers. Examples: EDI outsourcers such as SPS Commerce, True Commerce, GXS, Sterling, iTrade, OB10, or 1SYNC service.
- Optimizing inefficient processes. Example: Reduce cut N paste or the number of clicks needed to do a procedure.
- Archival of inactive customers and old data.
- Mini-training of I.T. and customer service, shipping, and accounts receivable users.
Factors for a successful EDI implementation
- Selection of EDI standards: One of the critical factors in the successful adoption of EDI is the availability of its standards. By using commercial ones and common message format, the development cost and the risk related to the new EDI application can reduce.
- Trading partner relationship: To increase your business cycle and avoid chargebacks its very important to understand the Trade Partner's requirements.
- Technology compatibility (hardware-software) among trading partners: Trading partners use multiple hardware platforms using different protocols, and problems may occur while expanding EDI externally among them.
- Availability of Value Added Networks (VAN): Companies can perform businesses with each other either by connecting directly to the relevant companies or with VAN. It acts as middleman among trading partners, who allows them to place purchase orders, invoices and remit payment automatically and securely.
- Integrate EDI with the business: Highlights opportunities to highlights opportunities to re-engineer the business process and reap further benefits. Integration usually consists of three key activities:
- The data analysis portion of the mapping
- Mapping via the EDI software
- Development of any custom interface programs or user exits
- Undertake data mapping: To exchange information data must in a format that complies with an EDI standard. Mapping describes how each element of the original data, such as an invoice relates to an "EDI transaction" meets with the rule used. The transaction software uses this mapping to translate the EDI transactions so the other organization can use them. The goal is to avoid the need for custom interfaces —especially custom edits per individual business partner. The more standardized the data formats, the better the system performance and the less need for specific programming.
- Execution of the pilot project: Once an organization has developed and tested its EDI system, further tests are conducted in pilot mode, enabling an organization to refine its system, show the benefits and ensuring that it can integrate with business partners.
Future of supply chain
Web services/API based integration
Web services are a platform for application integration, offering a model of plug-and-play, peer-to-peer, cross-platform collaboration.
Why web services?
One of the primary advantages of the Web services architecture is that it allows programming languages written on different platforms to communicate with each other in a standards-based way. Another advantage is that they work with standard Internet protocols – XML, HTTP, and TCP/IP and three core technologies.
- Simple Object Access Protocol (SOAP) - SOAP is the communication protocol for web services. It is a specification that defines the XML format for messages. It specifies an envelope for exchanging XML documents, an appropriate error-handling mechanism, and the binding to a specific transport protocol like HTTP. If you have a well-formed XML fragment enclosed within a pair of SOAP elements, you have a SOAP Message.
- Web Services Description Language (WSDL) - A WSDL file is an XML document, which describes a set of SOAP messages and how its exchanged. In other words, WSDL is to SOAP what IDL is to CORBA or COM.
- Universal Description Discovery and Integration (UDDI ) - Are the yellow pages of web services. A UDDI directory entry is an XML file that describes the business and the services it offers. It also includes several ways to search for the services you need to build your applications.
Benefits of web services
- Interoperability: Web services, at their core, are technologies designed to improve the interoperability between the many diverse application development platforms that exist today. Web services have the potential for seamless interoperability across heterogeneous systems, platforms, applications, and programming languages.
- Standardized Protocol: Web services uses standardized industry standard protocol for the communication. All the four layers (Service Transport, XML Messaging, Service Description and Service Discovery) uses the well-defined protocol in the web services protocol stack. This standardization of protocol stack gives the business advantages - cost reduction due to competition and increase in the quality.
- Service Transport Layer - This layer is responsible for transporting messages between applications. Currently, this layer includes Hypertext Transport Protocol (HTTP), Simple Mail Transfer Protocol (SMTP), File Transfer Protocol (FTP), and newer protocols such as Blocks Extensible Exchange Protocol (BEEP).
- XML Messaging Layer - This layer is responsible for encoding messages in a common XML format so that messages can be understood at either end. Currently, this layer includes XML-RPC and SOAP.
- Service Description Layer - This layer is responsible for describing the public interface to a specific web service. Currently, service description is handled via the WSDL.
- Service Discovery Layer - This layer is responsible for centralizing services into a common registry and providing easy publish/find functionality. Currently, service discovery is handled via UDDI.
- Platform Independent: Web services provide a standardized, platform-independent way for applications and organizations to share information and functionality. They are a practical, cost-effective solution for uniting information distributed between critical applications over the operating system, platform, and language barriers that were previously impassable. Since web services include HTTP and XML-based protocols including SOAP and WSDL, they are hardware, programming language, and operating system independent.
- Reducing Cost: All the above benefits add up to saving cost. Web services take advantage of ubiquitous protocols and the web infrastructure that already exists in every organization; they require little if any additional technology investment.
How API fits into the current logistics landscape?
Supply chains cannot afford to run on EDI technology, and the only way they can keep up is with the help of Application Programming Interface(API) technology.
APIs are the fastest and most seamless way to access data and make informed decisions in the supply chain. Forbes dubbed API as the “digital glue” holding our personal and professional worlds together, by allowing data to be transmitted from one system to another in nanoseconds1.
In fact, Gartner says “about three-quarters of Fortune—companies are using public APIs to grow their business”2. And still, most freight and transportation systems have yet to ride this wave.
API creates an automated supply chain that allows companies to identify risks, avoid issues before they occur. Set up of APIs is a one-time process, resulting in immediate access to an entire network of carriers – the code is structured and straightforward. It requires less than a week of time from start to finish to connect a shipper or 3PL into API network.
See the differences and complementarities between API data integration systems and EDI.
API creates an automated supply chain that allows companies to identify risks, avoid issues before they occur. Set up of APIs is a one-time process, resulting in immediate access to an entire network of carriers – the code is structured and straightforward. It requires less than a week of time from start to finish to connect a shipper or 3PL into API network.
| API | EDI |
|---|---|
| Application and user-oriented | Partner-oriented |
| Technical standards based | Industry standards based |
| Mobile device friendly | Business application friendly |
| Faster deployment | Medium length deployment |
| Ad hoc message formats. Driven mainly by the service implementer | Standardized message formats (orders, invoices, shipment notices). Driven mainly by standard bodies |
| Partner on-boarding has simple workflow | Partner on-boarding requires technical and business workflow |
| Usage conditions are defined unilaterally by the APIs | Business agreement are often required |
| Typically used for data and service exposure | Typically used for order-to-cash and similar supply chain cycles, as well as multi-chain interoperability |
| Value is in both partner relationship as well as service monetization | Value is in efficiency in partner relations |
| Owned by Chief Digital Officer and IT departments | Owned by supply chain and IT departments |
API creates an automated supply chain that allows companies to identify risks, avoid issues before they occur. Set up of APIs is a one-time process, resulting in immediate access to an entire network of carriers – the code is structured and straightforward. It requires less than a week of time from start to finish to connect a shipper or 3PL into API network.