Download Applied Distributed Computing: Message-Oriented Communication and more Lecture notes Data Communication Systems and Computer Networks in PDF only on Docsity!
Institute of Technology, UW-Tacoma
Communication
Wes J. Lloyd
Institute of Technology
University of Washington - Tacoma
TCSS 558: APPLIED DISTRIBUTED COMPUTING
Assignment #1 Questions
Assignment
Ch. 4 – Communications
Message-oriented communication: Zeromq, MPI, Message Queueing Systems Multicast communication November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
OBJECTIVES
Institute of Technology, UW-Tacoma 4.1 Foundations Protocols Types of communication 4.2 Remote procedure call 4.3 Message-oriented communication Socket communication Messaging libraries Message-Passing Interface (MPI) Message-queueing systems Examples 4.4 Multicast communication Flooding-based multicasting Gossip-based data dissemination November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
CHAPTER 4
Apa che A ct iveM Q CH. 4.3: MESSAGE- ORIENTED COMMUNICATION L13.
Institute of Technology, UW-Tacoma Ser vers execute 1st^ - 4 operations (socket, bind, listen, accept) Methods refer to C API functions Mappings across dif ferent libraries will var y (e.g. Java) November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
SOCKETS - 2
Operation Description socket Create a new communication end point bind Attach local address to socket (IP / port) listen Tell OS what max # of pending connection requests should be accept Block caller until a connection request arrives connect Actively attempt to establish a connection send Send some data over the connection receive Receive some data over the connection close Release the connection Socket: creates new communication end point Bind: associated IP and port with end point Listen: for connection-oriented communication, non-blocking call reser ves buffers for specified number of pending connection requests server is willing to accept Accept: blocks until connection request arrives Upon arrival, new socket is created matching original Server spawns thread, or forks process to service incoming request Server continues to wait for new connections on original socket November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
SERVER SOCKET OPERATIONS
Institute of Technology, UW-Tacoma Socket: Creates socket client uses for communication Connect: Server transport-level address provided, client blocks until connection established Send: Supports sending data (to: server/client) Receive: Supports receiving data (from: server/client) Close: Closes communication channel Analogous to closing a file stream November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
CLIENT SOCKET OPERATIONS
Sockets provide primitives for implementing your own TCP/UDP communication protocols Directly using sockets for transient (non-persisted) messaging is very basic, can be brittle Easy to make mistakes… Any extra communication facilities must be implemented by the application developer More advanced approaches are desirable E.g. frameworks with support common desirable functionality November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
SOCKET COMMUNICATION
Institute of Technology, UW-Tacoma Request-reply pattern Traditional client-server communication (e.g. RPC) Client: request socket (REQ) Server: reply socket (REP) Publish-subscribe pattern Clients subscribe to messages published by servers As in event-based coordination (Ch. 1) Supports multicasting messages from server to multiple Client: subscribe socket (SUB) Server: publish socket (PUB) November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
ZEROMQ - PATTERNS
Pipeline pattern (FIFO-queue) Analogous to a producer/consumer bounded buffer Producing processes generate results, push to pipe Consuming processes consume results, pull from pipe Producers: push socket (PUSH socket) Consumers: pull socket (PULL socket) Push- distributes messages to all pull clients evenly Consumers pull results from pipe and push results downstream November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
ZEROMQ – PATTERNS - 2
Institute of Technology, UW-Tacoma
Cloud services
Amazon Simple Queueing Service (SQS)
Azure service bus
Open source frameworks
Nanomsg
ZeroMQ
November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
QUEUEING ALTERNATIVES
MPI introduced – version 1.0 March 1994 Message passing API for parallel programming: su percomputer s Communication protocol for parallel programming for: Supercomputers, High Per formance Computing (HPC) clusters Point-to-point and collective communication Goals: high performance, scalability, portability Most implementations in C, C++, Fortran November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MESSAGE PASSING INTERFACE (MPI)
Institute of Technology, UW-Tacoma Very large library, v1.0 (1994) 128 functions Version 3 (2015) 440+ MPI data types: Provide common mappings November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MPI FUNCTIONS / DATAT YPES
MPI - no recovery for process crashes, network partitions Communication among grouped processes:(g rou pI D, p ro c e ss ID ) IDs used to route messages in place of IP addresses November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
COMMON MPI FUNCTIONS
Operation Description MPI_bsend Append outgoing message to a local send buffer MPI_send Send message, wait until copied to local/remote buffer MPI_ssend Send message, wat until transmission starts MPI_sendrecv Send message, wait for reply MPI_isend Pass reference to outgoing message and continue MPI_issend Pass reference to outgoing messages, wait until receipt start MPI_recv Receive a message, block if there is none MPI_irecv Check for incoming message, do not block!
Institute of Technology, UW-Tacoma Message-queueing systems Provide extensive support for persistent asynchronous communication In contrast to transient systems Temporally decoupled: messages are eventually delivered to recipient queues Message transfers may take minutes vs. sec or ms Each application has its own private queue to which other applications can send messages November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MESSAGE-ORIENTED-MIDDLEWARE
Enables communication between applications, or sets of processes User applications App-to-database To support distributed real-time computations Use cases Batch processing, Email, workflow, groupware, routing subqueries November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MESSAGE QUEUEING SYSTEMS:
USE CASES
Institute of Technology, UW-Tacoma Basic interface cont’d NOTIFY: install a callback function, for when msg is placed into a queue. Notifies receivers Queue managers: manage individual message queues as a separate process/library Applications get/put messages only from local queues Queue manager and apps share local network ISSUES: How should we reference the destination queue? How should names be resolved (looked-up)? Contact address (host, port) pairs Local look-up tables can be stored at each queue manager November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MESSAGE QUEUEING SYSTEMS
ARCHITECTURE
ISSUES: How do we route traffic between queue managers? How are name-to-address mappings efficiently kept? Each queue manager should be known to all others Message brokers Handle message conversion among dif ferent users/formats Addresses cases when senders and receivers don’t speak the same protocol (language) Need arises for message protocol converters “Reformatter” of messages Act as application-level gateway November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MESSAGE QUEUEING SYSTEMS
ARCHITECTURE - 2
Institute of Technology, UW-Tacoma November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MESSAGE BROKER ORGANIZATION
Plugins to convert messages between APPs Application-level Queues Message-queueing systems initially developed to enable legacy applications to interoperate Decouple inter-application communication to “open” messaging-middleware Many are proprietary solutions, so not ver y open e.g. Microsof t Message Queueing service, Windows NT 1997 Advanced message queueing protocol (AMQP), 2006 Address openness/interoperability of proprietary solutions Open wire protocol for messaging with powerful routing capabilities Help abstract messaging and application interoperability by means of a generic open protocol Suf fer from incompatibility among protocol versions pre-1.0, 1.0+ November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
AMQP PROTOCOL
Institute of Technology, UW-Tacoma Some examples: RabbitMQ, Apache QPid Implement Advanced Message Queueing Protocol (AMQP) Apache Kafka Dumb broker (message store), similar to a distributed log file Smart consumers – intelligence pushed off to the clients Stores stream of records in categories called topics Supports voluminous data, many consumers, with minimal O/H Kafka does not track which messages were read by each consumer Messages are removed after timeout Clients must track their own consumption (Kafka doesn’t help) Messages have key, value, timestamp Supports high volume pub/sub messaging and streams November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MESSAGE-ORIENTED-MIDDLEWARE
EXAMPLES:
Apa che A ct iveM Q CH. 4.4: MULTICAST COMMUNICATION L13.
Institute of Technology, UW-Tacoma Sending data to multiple receivers Many failed proposals for network-level / transport-level protocols to support multicast communication Problem: How to set up communication paths for information dissemination? Solutions: require huge management effort, human invention Focus shifted more recently to peer-to-peer networks Structured overlay networks can be setup easily and provide efficient communication paths Application-level multicasting techniques more successful Gossip-based dissemination: unstructured p2p networks November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MULTICAST COMMUNICATION
Overlay network Virtual network implemented on top of an actual physical network Underlying network The actual physical network that implements the overlay November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
NETWORK STRUCTURE
Institute of Technology, UW-Tacoma Stretch (Relative Delay Penalty RDP) for B to C routes: Overlay: BRbRaReEReRcRdDRdRc C = 73 Underlying: BRbRdRcC = 47 73 / 47 = 1. Tree cost: Overall cost of the overlay network Ideally would like to minimize network costs Find a minimal spanning tree which minimizes total time for disseminating information November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
MULTICAST TREE METRICS - 2
QUESTIONS
November 14, 2017 TCSS558: Applied Distributed Computing [Fall 2017] Institute of Technology, University of Washington - Tacoma L13.
Institute of Technology, UW-Tacoma EXTRA SLIDES 39
