WebSphere MQ

WebSphere MQ Series allows application programs to communicate with each other using messages and queues.This form of communication is also known as commercial messaging.

Communication Types

There are 2 types of communication models

Synchronous Communication model
Asynchronous Communication model

Synchronous application design model

Messaging — Synchronous application design model

Synchronous Communication

First Program ‘A’ prepares a message and places it on Queue 1. Then Program ‘B’ picks the message from Queue 1 and processes it.
Program ‘A’ and Program ‘B’ uses an API to place or pick messages from a queue. The WebSphere MQ API is called the Message Queue Interface (MQI).
When Program ‘A’ places a message on Queue 1, Program ‘B’ may not be running.
Queue keeps the message safe until Program ‘B’ starts and is ready to pick the message.
Similarly, when Program ‘B’ picks the message from Queue 1, Program ‘A’ may no longer be running.
So there is no need for the two programs to be executing at the same time when they are communicating with each other if we are using this model.
Now as we can see there is clearly a design issue, i.e. how long should Program’ A’ wait before continuing with other processing.
This design might be useful in some situations, but when the wait is too long, it is not.
Asynchronous communication is designed to work in those situations.

Asynchronous application design model

Asynchronous communication

When using asynchronous model, Program ‘A’ places the messages on Queue 1 for Program ‘B’ to process, but it is Program ‘C’, acting asynchronously to Program ‘A’, which gets the replies from Queue 2 and processes them.
Ideally, Program ‘A’ and Program ‘C’ should be part of the same application. You can see the flow of this activity in Figure 1-2.
The asynchronous model is natural for WebSphere MQ.
So Program ‘A’ is not blocked and can continue to place messages on Queue 1 as it doesn’t have to wait for a reply for each message.
Program ‘A’ can even continue to place messages on Queue 1 when Program ‘B’ fails. If so, then Queue 1 keeps the messages safely until Program ‘B’ is restarted.
In a different approach of the asynchronous model, Program ‘A’ can place a sequence of messages on Queue 1, optionally continue with some other processing, and then return to pick and process the replies by itself.
This property of WebSphere MQ, where communicating applications do not have to be active at the same time, is known as time independence.

Message types

WebSphere MQ uses four types of messages as mentioned
below:
Datagram    : A message for which no response is expected.
Request      : A message for which a reply is requested.
Reply          : A reply to a request message.
Report        : A message that describes an event, such as the occurrence of an error or a confirmation of the arrival of a delivery

Message queue and the queue manager

Message queue stores messages that are sent by programs.They belongs to the queue manager and are defined as objects.
When an application places a message on a queue, the queue manager ensures that the message is:

Stored safely
Recoverable
Delivered once, and once only, to the receiving application

This happens even if a message is to be delivered to a queue owned by another queue manager, this is known as the assure delivery property of WebSphere MQ.

Queue Manager

Queue manager (QM) is the software component that owns and manages queues. It provides messaging services for applications and ensures that messages are placed in the correct queue, routes messages to other queue managers, and processes messages through a common programming interface called the Message Queue Interface (MQI).
In the event of application or system outages QM can retain messages for future processing. Messages are retained in a queue until a successful completion response is received through the MQI.

Queue Manager vs Database Manager

Similarities between queue managers and database managers:

Queue managers own and control queues the same way database managers own and control their data storage objects.
Both Queue managers and database managers provide a programming interface to access data, and also provide security, authorization, recovery, and administrative facilities.

There are also important differences as mentioned below:

Databases are designed to provide long-time data storage with advanced search mechanisms, whereas queues are not designed for this.
A message on a queue generally indicates that a business process is incomplete; it might represent an unsatisfied request, an unprocessed reply, or an unread report.

Types of message queues

Several types of message queues exist like:

Local queue
Remote queue
Transmission queue
Initiation queue
Dead-letter queue

Local Queue

When a queue is owned by the queue manager to which the application program is connected it is termed as local queue.
It is used to store messages for programs that use the same queue manager. The application program need not run on the same machine as the queue manager.

Remote queue

When a queue is owned by a different queue manager it is termed as remote queue.
A remote queue is not a real queue, it is only the definition of a remote queue to the local queue manager. Programs cannot read messages from remote queues.

Transmission queue

When sending messages to queues that are owned by a different queue manager this queue is used as an intermediate step.
They are used internally by the queue manager channel initiator or we can say that they are transparent to the application.

Initiation queue

Queue manager writes (transparently to the programmer) a trigger message to this local queue when certain conditions are met on another local queue.

Example: When a message is placed into an empty message queue or in a transmission queue.

We have 2 WebSphere MQ applications to monitor initiation queues and read trigger messages, the trigger monitor, and the channel initiator.
The trigger monitor can start applications, depending on the message. The channel initiator starts the transmission between queue managers.

Dead-letter queue

A queue manager (QM) should be able to handle situations when a message cannot be deliver , for example:

The destination queue is full.
The destination queue does not exist.
The message puts have been inhibited on the destination queue.
The sender is not authorized to use the destination queue.
The message is too large.

When any of these conditions occurs, the message is written to the dead-letter queue. This queue is defined when the queue manager is created, and each QM should have one. It is used as a repository for all messages that couldn’t be delivered.

Channel

A channel is nothing but a logical communication link.
We should have a communications connection between each pair of communicating applications for conversational style of program-to-program communication.
Channels shield applications from the underlying communications protocols.

WebSphere MQ uses two kinds of channels:

Message channel
MQI channel

Message channel

It connects 2 queue managers through message channel agents (MCAs).
It is composed of 2 MCAs (a sender and a receiver) and a communication protocol. It is also unidirectional.
MCA transfers messages from a transmission queue to a communication link, and from a communication link to a target queue.
For bidirectional communication, it is necessary to define a pair of channels, consisting of a sender and a receiver.

MQI channel

An MQI channel connects a WebSphere MQ client to a queue manager.
Clients do not have a queue manager of their own.
An MQI channel is bidirectional.

In WebSphere MQ for z/OS, all channels run inside a separate process from the queue manager, called the Channel Initiator (CHINIT).

Interfacing with Online and Batch

Batch Processing

Batch processing is used when there is a lot of transactions affecting a high percentage of master file records and the response needed is not immediate, usually until the end of the week or month.
A good example of this in a large organization would be payroll processing, where nearly every master file record will be affected.
The data is collected over a period of time and verified (verified means input by someone else and then both inputs are compared by computer) and processed centrally.
The transactions are entered in batches by keyboard and stored in transaction files. These batches consist of thirty or so records which are given a batch control ID.
The batches are then run through a validation process and to make sure the batches balance a computed total is compared with a manually produced total. This helps to ensure that all data is entered without error or omission.
The actual updating of master files only takes place after verification and validation are complete. This means batch processing is often run overnight, unattended. A new master file is produced as a result of a batch processing run. The original master file is kept along with a previous version.
After processing the output is produced, and is usually printed media such as pay slips or invoices.

Online Transaction Processing

Online processing means users directly enter information online , it is validated and updated directly onto the master file, no new file is created in this case. Therefore, there is near immediate input process, and output.
Imagine a cash dispenser transaction or booking a holiday at a travel agents or over the Internet. Compared with batch processing the number of transactions will be few.
Online comes in many different flavors such as centralized, distributed, time-share etc and the choice of architecture will depend on cost, speed needed, number of users, number of transactions and time needed for a response.

Online and Batch Interfacing

With mainframe modernization, there are some software products that extend the capability of these online transaction programs.
These software products provide interface to web applications and communicate with these online transaction programs.
This technique leverages the existing assets and make them more relevant in current business environments.

Message thru EDI and XML

What is EDI?

Def. 1 – The exchange of documents in standardized electronic form, between organizations, in an automated manner, directly from a computer application in one organization to an application in another.
Def. 2 – The electronic data interchange process is the computer-to-computer exchange of business documents between companies. EDI replaces the faxing and mailing of paper documents.

Benefits of EDI

Data can be transferred in faster way
Increases the productivity
Saves unnecessary re-capturing of data
Reduce manual errors and improves accuracy
Less time wasted on exception handling, hence improves business process
Saves much cost in compare to traditional message system
Data can be sent in secured way
Eliminates the use of paper, helps preserves the environment
Transactions support is faster and reduces inventory levels

Drawbacks of EDI

EDI is expensive and initial setup is time consuming
Lack of expert EDI consultants or IT staff to make it work
Extensive telecommunication capability will be required

How EDI works

Sender and Receiver in EDI

EDI is a batch process in which transactions are grouped together in to one or more files, all transmitted at one time
Company A is sender(outbound) and Company B is receiver (Inbound), both company A and B will become senders and receivers through out the relationships
In most cases one company will be driver of the EDI process and other company will follow the process
The driver can be a customer, organization etc.. And follower can be supplier, association who can deal with the organization
The driver is the responsible to publish implementation guide in the whole process

Data Transport

Data transport is the process of electronically transferring the files to/from your trading partners
Traditionally VAN(Value Add Network) will be used to transfer the data between the vendors, now a days internet will be highly used to connect the VAN and send the data at point to point to the vendor location
The type of data transport method to be used will be decided by the driver.

Data Translation

Each and every EDI transaction is a raw data and it is meaningless until it process thru and EDI translator
Data translation is the process of interpreting the EDI data to acceptable EDI standards and perform checks before it reaches to intended business application
EDI translator will perform the following functions
Compliance Check – Checks the EDI standards
Control Number check – To track the EDI transmission documents to make sure duplicates are not sent
Trading partner management – Maintains EDI trading relation ships with partners
Tracking and auditing – Track the EDI transactions that did not make it to the business application
Document Repository – Store all incoming EDI and Non-EDI transactions

Data Transformation

Data translation and transformation looks same but the concept wise they are different.
As explained in data translation section, it is the process of interpreting the EDI data to ensure to confirm EDI standards. Data Transformation is the process of taking data from an inbound record layout and moving it to an outbound record layout and this is also known it as mapping, any data format (XML, EDI,TXT etc.) can be transformed but only EDI data is mapped.

Data Integration

Any business application will accept an EDI transaction in a raw format for processing.
There should be an integration software in the middle that will map EDI transaction in to the format that the business application will accept.
The business application will provide the doorway to allow a transaction to enter in one of 2 below ways.

The integration software will walk thru the door and interact with the application directly
The integration software will hand off the transaction at the door and business application will take over.

XML – eXtensive Markup Language

As the name suggests, XML is a markup language.
XML allows you to create your own tags to describe the data between them. You’re not particularly interested in how the data will be presented.
Your main focus is ensuring that the data is well organized within descriptive tags (or elements). This is because XML is primarily used for data storage and transfer purposes – not for presentation purposes.

Benefits of XML

Easy data exchange – One of the good thing about XML is that it can allow easy sharing of data between different applications – even if these applications are written in different languages and reside on different platforms.
Self-describing data – When you look at an XML scripting, it is very easy to figure out the what content of data is transferred.
Create your own languages – XML allows you to specify your own markup language for your own specific purpose.

XML Viewers

You can see the XML content using any of the below applications.

Notepad
Internet Explorer
Mozilla Firefox

XML Editors

An XML editor will make your life easier when coding XML, also it will help you to prevent from making errors.
The below are some XML editors can be used.

XML Notepad
XML Cooktop
XML Pro
XML Spy

XML Syntax

If you include an XML declaration, it must be the first item in your document. The below is the syntax for XML declaration.
<?xml version=”1.0″ encoding=”UTF-8″ standalone=”no”?>
Each XML document must have one root element. All other elements must be contained within the root element.
<root>
<child>Test Data</child>
<child>Test Data</child>
</root>

XML Elements

XML elements are represented by tags. Elements usually consist of an opening tag and a closing tag
Opening tags consist of <, followed by the element name, and ending with >.
Closing tags are the same but have a forward slash inserted between the less than symbol and the element name.
For Example:
<tag>Test Data</tag>
Note: All elements must close properly and all tags are case sensitive.
Things to remember:

Element names can contain any character
Element names must not contain spaces
Element names must not begin with a number or punctuation character Element names must not start with the letters xml

XML Attributes

XML elements can also contain attributes.
You use attributes within your elements to provide more information about the element. These are represented as name/value pairs.
For Example:
<tag attribute=”value”>Test Data</tag>

XML Entities

XML entities allow you to use text to refer to a data item, instead of using the data item itself.
Entity Syntax:
>
The reason why we use entity in XML is, for example to represent 10>5. But system understand “>” is closing of tag, so to avoid this confusion we use entities.
Some of predefined entities

<	<
>	>
&	&
"	“
'	'

EDI or XML implementation process

Understanding of Electronic data Exchange.
Evaluate the business situation.
Identify trading partners (sender and receiver).
Develop project plan.
Define Electronic data Exchange process.
Choose Electronic data Exchange translation software.
Mapping data To and From Electronic data Exchange.
Application development, automation and Integration.
Establishing Trading partners.
Electronic data Exchange testing.
Electronic Data Exchange Implementation.

About The Author

Trenovision

Subscribe us to unlock content