Internet and intranet protocols and applications
1 / 42

Internet and Intranet Protocols and Applications - PowerPoint PPT Presentation

  • Uploaded on

Internet and Intranet Protocols and Applications. Lecture 14 Introduction to MQSeries And Asynchronous Messaging May 1, 2002 Joseph Conron Computer Science Department New York University What is MQSeries?.

I am the owner, or an agent authorized to act on behalf of the owner, of the copyrighted work described.
Download Presentation

PowerPoint Slideshow about 'Internet and Intranet Protocols and Applications' - ziv

An Image/Link below is provided (as is) to download presentation

Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author.While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server.

- - - - - - - - - - - - - - - - - - - - - - - - - - E N D - - - - - - - - - - - - - - - - - - - - - - - - - -
Presentation Transcript
Internet and intranet protocols and applications

Internet and Intranet Protocols and Applications

Lecture 14

Introduction to MQSeries

And Asynchronous Messaging

May 1, 2002

Joseph Conron

Computer Science Department

New York University

What is mqseries
What is MQSeries?

  • A middleware product that implements a messaging and queuing framework.

  • Middleware - an intermediate software component that bridges dissimilar computing environments.

    • Unix, MVS, OS/400 Tandem, VMS, NT, etc.

    •  SNA, NetBios, TCP/IP

    • Cobol, C, JAVA

Messaging and queueing
Messaging and Queueing

  • Messaging - programs communicate by sending data in messages rather than by calling each other directly.

  • Queuing - messages are put on queues in storage, eliminating the need for programs to be logically connected.

  • A messaging and queuing framework is inherently ASYNCHRONOUS!

Asynchronous vs synchronous communications
Asynchronous vs. Synchronous Communications

  • Synchronous: App sends request, then blocks until request is processed.

    • Requires service available at EXACTLY same time as client needs service.

  • Asynchronous: App sends request and checks at some future time if complete.

    • Service need not be available when client sends request

Synchronous good bad

Easy to program

Outcome is known immediately

Error recovery easier (usually)

Better real-time response (usually)

Service must be up and ready

requestor blocks, held resources are “tied up”

Usually requires connection-oriented protocol

Synchronous: good/bad



Internet and intranet protocols and applications

Asynchronous: good/bad



  • Requests need not be targeted to specific server.

  • Service need not be available when request is made

  • No blocking, so resources could be freed

  • Could use connectionless protocol

  • Response times are unpredictable

  • error handling usually more complex

  • Usually requires connection-oriented protocol

  • Harder to design apps?

Messaging vs procedure calls
Messaging vs. Procedure Calls

  • As programmers, many of us think procedurally, so using procedures is “natural” extension of how we think.

  • Messages are an abstract concept: harder for us to conceptualize relationship between actions and messages!

  • But Wait!

    • Something good happens when we use abstractions!

Internet and intranet protocols and applications

<Soap Box>

We are free to concentrate on the design of the application itself.

We are no longer concerned with details of the environment.

Our application is suddenly portable and to some degree, extensible.

</Soap Box>

A brief history of mqseries
A Brief History of MQSeries

  • 1992

    • Systems Strategies (SSI) develops ezBridge, a messaging and queuing product for VMS, Tandem, and Unix

    • IBM announces Networking Blueprint defining three standard APIs for program to program communication: CPI-C, RPC, MQI

  • 1992-3

    • State Street Bank (Boston) evaluates IBM messaging product (code name “Victory”) for IBM CICS/ESA and SSI’s ezBridge on VMS and Tandem.

      “State Street Bank would like to announce the wedding of

      IBM and Systems Strategies!”

  • 1993

    • IBM buys intellectual property rights for ezBridge from SSI

The bride s wedding preparation what ssi had to do
The Bride’s Wedding Preparation(What SSI had to do)

  • Implement IBM Channel Protocol over TCP/IP and LU6.2 (more about channels later).

  • Implement the MQI interface functions (MQCONN, MQOPEN, MQPUT, MQGET, MQCOMMIT, MQCLOSE, MQDISC).

  • Implement MQI Error Semantics (failure conditions should look the same on all systems)

Mqseries ezbridge integration objectives
MQSeries/ezBridge Integration Objectives

  • Applications written in C using ezBridge on VMS and Tandem had to exchange messages with applications written in (COBOL?) Using MQI under CICS/ESA.

  • System intercommunication using channel protocol over TCP/IP and LU6.2 (that is, A VMS system had to “look” to IBM machine just like another IBM system!).

Mqseries platform rollout
MQSeries Platform Rollout

  • Initially, IBM’s version of MQSeries ran only on mainframe

    • (CICS/ESA, IMS/ESA, and eventually VSE).

  • SSI version (called MQSeries Version 1) initially released on:

    • VMS, Tandem, AS/400, and Unix (SCO, UnixWare).

  • 1994/1995 -IBM releases the first three “distributed” platforms:

    • AIX, OS/2, and AS/400.

    • The AIX version becomes the “reference port”.

  • Over time, IBM replaced SSI versions with “ports” of its reference system.

  • Today - MQSeries runs on over 35 platforms

How messaging queuing works
How Messaging & Queuing Works

Programs communicate by putting messages on queues. Here, program A puts a message on Queue1, which is read by program B.

Note: A and B need not be on the same machine!

How messaging queuing works 2
How Messaging & Queuing Works (2)

Communication can be one-way or two-way. Here, A sends to B on Queue1, and B responds to A on Queue2

Messaging and queuing characteristics
Messaging and Queuing Characteristics

  • Three key facts about Messaging and Queuing differentiate it from other communication styles:

    1) Communicating programs can run at different times.

    2) There are no constraints on application structure.

    3) Programs are insulated from environmental differences.

  • Let’s examine each of these characteristics in more detail.

Applications can run at different times
Applications Can Run at Different Times

Either program can be unavailable

Key Concept:message queue exists independently from programs that use them!

No constraints on application structure
No Constraints on Application Structure

There can be a one to many relationship between applications

Or a many to one relationship between applications

Applications shielded from environmental differences
Applications Shielded from Environmental Differences

  • Don’t care what OS is used.

  • Don’t care what language they’re written in

  • Don’t care what the underlying communication protocol is used.

Applications shielded from environmental differences1
Applications Shielded from Environmental Differences


Programmatic API

Queue Manager

Queue Manager

Communications using Message Channels

Message Queue

Mqseries objects queue manager
MQSeries Objects: Queue Manager

  • Queue Manager

    • Controls access to queues:

      • administration (create, delete, etc)

      • usage (Put, Get)

    • serves as transaction (syncpoint) coordinator for all queue operations.

    • Accessed through the Message Queue Interface (MQI)

    • Queue Managers have names (identities) that are UNIQUE in a network (like host names).

Mqseries objects queues
MQSeries Objects: Queues

  • MQSeries defines four types of queues. A queue instance is fully qualified by its queue manager and queue name.

    • Local Queue - an actual queue for which storage is allocated.

    • Remote Queue - a definition of a queue on a different queue manager (acts somewhat like a pointer)

    • Alias Queue - another name for a local or remote queue. Typically used to switch queue destinations without modifying program code.

    • Model Queue - a template whose properties are copied when creating a new dynamic local queue (“ create queue xxx “like” queue yyy).

Mqseries queues properties
MQSeries Queues: Properties

  • Maximum Message Size

  • Maximum Queue Depth

  • High/Low Factors

  • Enable/Disable Put or Get

  • Persistent/Not Persistent

Mqseries queues events and triggering
MQSeries Queues: Events and Triggering

  • Local queues can generate events (messages) under certain conditions (like queue full).

  • These “event” messages can be used to “trigger” the execution of a program.

  • These events are called trigger messages. The queue on which they are put is called an Initiation Queue.

Mqseries objects processes
MQSeries Objects: Processes

  • Process defines an application to an MQSeries queue manager. A process definition object is used for defining applications to be started by a trigger monitor.

  • A trigger monitor is a program that listens on an initiation queue and executes commands named in Process definitions.

  • Triggering is useful when you don’t want to deploy long-running programs.

Mqseries objects message channels
MQSeries Objects: Message Channels

  • provide a communication path between two queue managers on the same, or different, platforms.

  • A message channel can transmit messages in one direction only. If two-way communication is required between two queue managers, two message channels are required.

  • Question: why make message channels one-way?

Mqseries objects message channels 2
MQSeries Objects: Message Channels(2)

  • Types of message channels:

    • Sender - initiates connection to Receiver

    • Server - Accepts request to start from requester, then becomes Sender

    • Receiver - Passive; waits for initiation sequence form Sender

    • Requester - Active at start, then becomes Receiver

    • Cluster-sender (used amongst Cluster Queue Managers)

    • Cluster-receiver (ditto)

Mqseries message channel concepts
MQSeries: Message Channel Concepts

  • The Sender side of the session is the “transaction coordinator”.

  • Message channels implement a protocol that includes a commitment protocol.

  • Channels recover from failure by agreement: they must agree on the last committed unit of work [would this be harder if channels were bi-directional??]

  • Message Chanels are implemented by programs called Message Channel Agents (MCA)

How messages move across channels
How messages move across channels

(1) Application puts message on transmission queue

(4) Message is available on local queue for applications

(2) Sender MCA gets message and sends to partner MCA

(3) Receiver MCA puts message on target queue

Mqseries assured delivery
MQSeries: Assured Delivery

  • If queues are persistent, and message is persistent, then MCAs will eventually deliver the message to the target queue, and target application will get it!

  • MCAs have recoverable state - they’re STATEFUL - and a connection-oriented protocol.

  • Message is not removed from xmit queue until partner MCA confirms placement on target queue

Mqseries objects messages
MQSeries Objects: Messages

  • Messages consist of:

    • Header (MQMD)

      • Used by Queue Manager and application to handling properties of the message

    • User Data

      • The application-to-application data (“payload”)

      • transparent to MQSeries

Mqseries messages properties
MQSeries Messages: Properties

  • Destination Queue

  • Reply Queue name

  • Time to live (expiry)

  • Format

  • Correlation ID

  • Persistence

  • “Report” options

Mqseries messages properties 2
MQSeries Messages: Properties (2)

  • Messages can be individually designated persistent or non-persistent(persistent messages are logged to enable recovery)

  • Correlation ID - select which message to get from queue

  • Segmented Messages - allows ending of VERY LARGE messages (> 100 MB)

Mqseries messages message types
MQSeries Messages: message types

  • Request - used by one program to ask another program for something (usually data). A request message needs a reply.

  • Reply - used in response to a request message.

  • A one-way message, as you would expect, doesn’t need a reply, though it can carry data.

  • A report message is used when something unexpected occurs, or to give extra information like:

    • message delivered to target queue

    • message taken by application

    • message not deliverable

    • message exceeded time-to-live

Mqseries messages units of work and transactions
MQSeries MessagesUnits of Work and Transactions

  • Messages are added and removed from queues in Units of Work

  • The smallest Unit of Work is one message.

  • Units of work are atomic.

  • When an app reads a message from a queue, a message “appears” to have been removed, but in fact, it is still in storage until the app “commits” the unit of work.

Mqseries transaction support
MQSeries: Transaction Support

Unit of recovery -a piece of work that changes data from one point of consistency to another.

Syncpoint - A point of consistency (also called a or commit point). It is a moment at which all the recoverable data that an application program accesses is consistent.

Mqseries transaction support 2
MQSeries: Transaction Support (2)

  • Applications are responsible for delimiting the beginning and end of a transaction. How can messaging be coordinated with a data base update?

  • MQSeries is XA compliant and can operate with other XA compliant systems as either a transaction manager (coordinator) or resource manager (particpant).

    Some examples: Sybase, DB2, Oracle.

Mqseries logging and recovery
MQSeries: Logging and Recovery

  • All operations that affect the “state” of the Queue Manager and its objects are logged to a log file.

  • What is “state”?

    • Object definitions (queue manager, queues, processes, channels, etc)

    • Queue content (messages)

    • What about message channel state?

  • Message channel states are logged separately by each channel

  • Mqseries logging and recovery 2
    MQSeries: Logging and Recovery (2)

    • Two forms of Logging

      • Circular – log records are written sequentially across several files, then “wrap” back to the first file.

      • Linear - log records are written sequentially across files. New files are allocated as current files fill. No automatic reuse of file space!

    • Problem: Length (in time) of longest running transaction vs amount of writes to log determines size of log needed.

    Mqseries logging and recovery 3
    MQSeries: Logging and Recovery (3)

    • Observations:

      • Circular logging is easy to manage, but is fatal if log is damaged (hard to backup circular logs!).

      • Linear logging is hard to maintain but provides for archiving of previous logs (still a problem if “current” log is damaged).

    Mqi the mqseries programming interface
    MQI: The MQSeries Programming Interface

    MQCONN Connect to queue manager

    MQDISC Disconnect from queue manager

    MQOPEN Open queue

    MQCLOSE Close queue

    MQPUT Put message

    MQGET Get message

    Mqi the mqseries programming interface 2
    MQI: The MQSeries Programming Interface (2)

    MQPUT1 Put one message (implied open/put/close)

    MQBEGIN Begin unit of work

    MQCMIT Commit

    MQBACK Back out

    MQINQ Inquire about queue attributes

    MQSET Set queue attributes

    Mqseries language support
    MQSeries: Language Support

    • C

    • C++

    • Cobol

    • JAVA (native and JMS)

    • PL/I

    • System 390 Assembler

    • TAL (Tandem)

    • Visual Basic

    • For More Information: