[esnacc-dev] [PATCH] doc: Add a sample C++ application for network discovery
Aaron Conole
aconole at bytheb.org
Thu Nov 17 16:07:53 UTC 2016
Documentation is severely lacking. It would be a great benefit to the project
to show a few sample applications for exchanging data. This commit adds a new
simple application suitable for illustrating some of the techniques and power
of developing with esnacc.
Signed-off-by: Aaron Conole <aconole at bytheb.org>
---
doc/developer-guide-sample-app.md | 518 ++++++++++++++++++++++++++++++++++++++
doc/developer-guide.md | 1 +
2 files changed, 519 insertions(+)
create mode 100644 doc/developer-guide-sample-app.md
diff --git a/doc/developer-guide-sample-app.md b/doc/developer-guide-sample-app.md
new file mode 100644
index 0000000..ea9e0c1
--- /dev/null
+++ b/doc/developer-guide-sample-app.md
@@ -0,0 +1,518 @@
+Developer's Guide: Sample Communications Application in C++
+===========================================================
+
+## Contents
+
+- [Introduction](#introduction)
+- [Protocol](#protocol)
+- [Implementation](#implementation)
+- [Using](#using)
+
+## Introduction
+
+### In-Depth Application
+
+The following application will show a sample bonjour-like discovery service,
+which can be used for network discovery across various multicast domains. The
+purpose of this document will be to show the design of the protocol, as well
+as a sample implementation demonstrating the power of eSNACC and ASN.1
+
+
+### What you will need
+
+It is assumed that the following are true:
+
+- You are on a system with an installed eSNACC distribution, at least 1.8
+- You have access to a c++ compiler, usually G++
+- You have a multicast network available and multiple test machines (VMs?)
+
+
+### Where to get help
+
+As always, consult the mailing lists. Specifically, `dev at lists.esnacc.org` or
+`bugs at lists.esnacc.org` to report issues.
+
+
+## Protocol
+
+### Goals
+
+- Allow for automatic discovery of network elements.
+- Allow for registration of services with the discovery agents
+- Allow 'hop' nodes to query for discovery status
+- Exchange data on the discovery network, both unicast and multi-cast.
+
+### High level overview
+
+The discovery mechanism is made up of two separate but important functions.
+
+The low-level network transmission information and a higher-level series of
+state machines which coordinates services. Discovery nodes will advertise
+themselves to peer entities, and broadcast their service definitions. They
+must do this in a host-machine agnostic way.
+
+The network mode of transmission must support both traditional unicast
+messaging, as well as multicast group messaging. It may be independent of
+Internet Protocol version, but at least must support either ipv4 or ipv6
+addresses.
+
+The nodes must share their information via a series of hello and status
+messages, with a simple pilot 'hello' message being the main bulk of network
+traffic exchanged in the multicast group.
+
+### ASN.1 Definition
+
+```
+NodeDiscovery DEFINITIONS AUTOMATIC TAGS ::=
+BEGIN
+
+ OperatingSystem ::= ENUMERATED
+ {
+ unknown,
+ linux,
+ windows,
+ freebsd,
+ openbsd,
+ netbsd,
+ ...
+ }
+
+ Time ::= SEQUENCE
+ {
+ sec INTEGER,
+ nsec INTEGER
+ }
+
+ IPv4Address ::= INTEGER
+ IPv6Address ::= OCTET STRING
+
+ AddressType ::= CHOICE
+ {
+ v4Addr IPv4Address,
+ v6Addr IPv6Address
+ }
+
+ TcpPort ::= INTEGER (1024..65535)
+ UdpPort ::= INTEGER (1024..65535)
+
+ PortChoice ::= CHOICE
+ {
+ tcp TcpPort,
+ udp UdpPort
+ }
+
+ Host ::= SEQUENCE
+ {
+ addressType AddressType,
+ systemHostName VisibleString OPTIONAL
+ }
+
+ NodeInformation ::= SEQUENCE
+ {
+ hostInformation Host,
+ discoveryUUIDName VisibleString,
+ operatingSystem OperatingSystem OPTIONAL,
+ currentTimeSinceEpoch Time OPTIONAL,
+ uptime Time,
+ ...
+ }
+
+ Service ::= SEQUENCE
+ {
+ listenerAddress AddressType,
+ portInformation PortChoice OPTIONAL,
+ serviceName OCTET STRING
+ }
+
+ DiscoveryHello ::= SEQUENCE
+ {
+ hostDescription Host,
+ discoveryUUIDName VisibleString
+ }
+
+ DiscoveryStatus ::= SEQUENCE
+ {
+ hostDescription NodeInformation,
+ providedServices SEQUENCE OF Service OPTIONAL,
+ requestedCommand OCTET STRING OPTIONAL,
+ commandResponse OCTET STRING OPTIONAL,
+ ...
+ }
+
+ DiscoveryMessage ::= CHOICE
+ {
+ hello DiscoveryHello,
+ status DiscoveryStatus,
+ ...
+ }
+
+END
+```
+
+
+### Description
+
+The discovery agents will exchange `DiscoveryMessage` objects via the discovery
+network. The discovery network will be a multi-cast group, with group address
+and udp port to be defined by the network administrator. All `Host`
+participants will advertise their node address, which is the preferred address
+to reach the that particular node.
+
+
+
+## Implementation
+
+The implementation shown here will be a mostly-complete C++ application, ready
+to deploy. Some details will need to be expanded to make the application more
+usable.
+
+Let's start with the basics, a database of discovered nodes. We really only
+need to decide on the best ways of storing and indexing into that data. The
+following represents a simple in-memory storage for the database.
+
+```c++
+#include <map>
+#include <memory>
+#include <vector>
+#include <time.h>
+#include <signal.h>
+
+#include "NodeDiscovery.h"
+
+struct NodeInfo {
+ SNACC::NodeInformation node;
+ SNACC::Time lastHeard;
+
+ NodeInfo(SNACC::NodeInformation &n, SNACC::Time &t) : node(n),
+ lastHeard(t) {}
+};
+
+class NodeDatabase
+{
+ typedef std::shared_ptr<NodeInfo> node_t;
+ std::vector<node_t> m_oNodes;
+ std::map<std::string, node_t> m_oNodesByHostName;
+ std::map<std::string, node_t> m_oNodesByDiscoveryUUID;
+
+public:
+ class DBAddObserverCallback
+ {
+ public:
+ DBAddObserverCallback() {}
+ virtual ~DBAddObserverCallback() {}
+
+ virtual void callback(node_t node) = 0;
+ };
+
+protected:
+
+ std::vector<DBAddObserverCallback*> m_callbacks;
+
+ template <typename T>
+ node_t RetrieveNodeInKeyedDatabase(const T &key,
+ const std::map<T, node_t> &db) const {
+ if (db.find(key) != db.end()) {
+ return db.find(key)->second;
+ }
+ return node_t(0);
+ }
+
+public:
+
+ NodeDatabase() : m_oNodes(), m_oNodesByHostName(),
+ m_oNodesByDiscoveryUUID(), m_callbacks()
+ {
+ // nothing
+ }
+
+ ~NodeDatabase() { }
+
+ void AddCallback(DBAddObserverCallback *cb)
+ {
+ m_callbacks.push_back(cb);
+ }
+
+ node_t RetrieveNodeByHostName(const std::string &host) const
+ {
+ return RetrieveNodeInKeyedDatabase(host, m_oNodesByHostName);
+ }
+
+ node_t RetrieveNodeByUUID(const std::string &uuid) const
+ {
+ return RetrieveNodeInKeyedDatabase(uuid, m_oNodesByDiscoveryUUID);
+ }
+
+ void AddNode(SNACC::NodeInformation &node)
+ {
+ SNACC::Time currentTime;
+ {
+ struct timespec ts;
+ if (::clock_gettime(CLOCK_MONOTONIC, &ts)) {
+ perror("clock_gettime");
+ raise(SIGABRT);
+ }
+ currentTime.sec = ts.tv_sec;
+ currentTime.nsec = ts.tv_nsec;
+ }
+ /* first - is this node already added */
+ if (node.hostInformation.systemHostName) {
+ if (RetrieveNodeByHostName(*node.hostInformation.systemHostName)) {
+ RetrieveNodeByHostName(*node.hostInformation.systemHostName)->
+ lastHeard = currentTime;
+ return;
+ }
+ }
+
+ if (RetrieveNodeByUUID(node.discoveryUUIDName)) {
+ RetrieveNodeByUUID(node.discoveryUUIDName)->lastHeard = currentTime;
+ return;
+ }
+
+ if (!node.discoveryUUIDName.length()) {
+ // no discovery uuid!? bad!
+ return;
+ }
+
+ /* does not exist - see if we can add it to the database */
+ node_t newNode(new NodeInfo(node, currentTime));
+ m_oNodes.push_back(newNode);
+
+ if (node.hostInformation.systemHostName) {
+ m_oNodesByHostName[*node.hostInformation.systemHostName] =
+ newNode;
+ }
+
+ m_oNodesByDiscoveryUUID[node.discoveryUUIDName] = newNode;
+ for (std::vector<DBAddObserverCallback*>::iterator c =
+ m_callbacks.begin();
+ c != m_callbacks.end(); ++c) {
+ (*c)->callback(newNode);
+ }
+ }
+
+ void CopyNodes(std::vector<NodeInfo> &nodes)
+ {
+ // copies all the node information
+ for (std::vector<node_t>::iterator i = m_oNodes.begin();
+ i != m_oNodes.end(); ++i) {
+ nodes.push_back(*(*i));
+ }
+ }
+};
+```
+
+In the database implementation, we do a few things. Whenever a node is passed
+to the Add routine, if it exists we update it's last-heard time. If not, we
+will add it to the database, and call the various observer callbacks.
+
+With that database, we can store nodes which we learn 'on-the-fly'. We'll
+need a mechanism for receiving discovery data. Luckily, *esnacc* does provide
+a way of doing this:
+
+```c++
+class NodeDiscoveryCB : public SNACC::StreambufCallback
+{
+ NodeDatabase *db;
+public:
+ NodeDiscoveryCB(NodeDatabase *d) : db(d) {}
+ ~NodeDiscoveryCB(){}
+
+ SNACC::StreambufCallback::Streambuf_CB_RESULT
+ callback(SNACC::StreambufCallback::Streambuf_CB_STATUS status,
+ std::streambuf *stream)
+ {
+ if (status == CB_READ_OK) {
+ if (stream) {
+ SNACC::AsnBuf b(stream);
+ SNACC::AsnLen l;
+ SNACC::DiscoveryMessage msg;
+ try {
+ msg.BDec(b, l);
+ } catch (SNACC::SnaccException &e) {
+ // should we abort here? for now yes
+ raise(SIGABRT);
+ }
+ switch (msg.choiceId) {
+ default:
+ raise(SIGABRT);
+ break; // never reached
+ case SNACC::DiscoveryMessage::helloCid:
+ {
+ SNACC::NodeInformation node;
+ node.hostInformation = msg.hello->hostDescription;
+ node.discoveryUUIDName = msg.hello->discoveryUUIDName;
+ db->AddNode(node);
+ }
+ break;
+ case SNACC::DiscoveryMessage::statusCid:
+ {
+ // update / refresh the database
+ db->AddNode(msg.status->hostDescription);
+ // likely in response to a request so just print everything
+ std::cout << *(msg.status) << std::endl;
+ }
+ break;
+ };
+ }
+ }
+ }
+};
+```
+
+This callback can be used in conjunction with a streambuf monitor to
+automatically receive and process messages from the discovery network.
+
+The only thing left is to add a simple transport for the network - here we
+can use socket calls and setup an IPv4 IGMP socket and add the socket to
+an existing streambuf monitor:
+
+```c++
+#include <sys/socket.h>
+#include <netinet/in.h>
+#include <arpa/inet.h>
+
+NodeDiscoveryCB gCallback;
+
+int get_multicast_socket(unsigned short port)
+{
+ struct sockaddr_in addr;
+ int fd, nbytes;
+ socklen_t addrlen;
+ struct ip_mreq mreq;
+
+ if ((fd = socket(AF_INET, SOCK_DGRAM, 0)) < 0) {
+ perror("socket");
+ raise(SIGABRT);
+ }
+
+ // set up addresses
+ bzero(&addr, sizeof(addr));
+ addr.sin_family = AF_INET;
+ addr.sin_addr.s_addr = htonl(INADDR_ANY);
+ addr.sin_port = htons(port);
+
+ if (bind(fd, (struct sockaddr *) &addr, sizeof(addr)) < 0) {
+ perror("bind");
+ raise(SIGABRT);
+ }
+
+ mreq.imr_multiaddr.s_addr = inet_addr("224.0.0.1");
+ mreq.imr_interface.s_addr = htonl(INADDR_ANY);
+ if (setsockopt(fd, IPPROTO_IP, IP_ADD_MEMBERSHIP, &mreq, sizeof(mreq))< 0) {
+ perror("setsockopt");
+ raise(SIGABRT);
+ }
+
+ return fd;
+}
+
+void JoinStreambufMonitorToPort(int fd, SNACC::StreambufMonitor *mon,
+ unsigned short p)
+{
+ if (fd < 0) fd = get_multicast_socket(p);
+ SNACC::AsnFDBuf *fdb = new SNACC::AsnFDBuf(fd, true);
+ mon->push(fdb, gCallback, false);
+}
+
+```
+
+And tie the whole thing together with:
+
+```c++
+#include <unistd.h>
+
+int main(int argc, char *argv[])
+{
+ SNACC::StreambufMonitor mon;
+ int fd = get_multicast_socket(44321);
+ JoinStreambufMonitorToPort(fd, &mon, 44321);
+ while (1) {
+ mon();
+ sleep(5);
+
+ SNACC::DiscoveryMessage msg;
+ msg.choiceId = SNACC::DiscoveryMessage::helloCid;
+ msg.hello = new SNACC::DiscoveryHello;
+ msg.hello->hostDescription.addressType.choiceId =
+ SNACC::AddressType::v4AddrCid;
+ msg.hello->hostDescription.addressType.v4Addr =
+ new SNACC::IPv4Address(0x01);
+
+ std::stringstream s;
+ SNACC::AsnBuf b(s.rdbuf());
+ msg.BEnc(b);
+
+ struct sockaddr_in addr;
+ addr.sin_family = AF_INET;
+ addr.sin_port = htons(44321);
+ addr.sin_addr.s_addr = inet_addr("224.0.0.1");
+ sendto(fd, s.str().c_str(), s.str().length(), 0,
+ (struct sockaddr *)&addr, sizeof addr);
+ }
+ return 0;
+}
+```
+
+The short explanation is as follows:
+
+1. When the binary starts, it creates (a) a socket monitor, (b) a multi-cast
+ socket, and (c) joins them together.
+
+2. During the main loop, (b) is monitored for data.
+
+3. If data is detected, the callback is invoked flagging read-readiness. The
+ callback attempts to decode a packet on the wire. NOTE: no sanity checking
+ is done to protect against malicious users.
+
+4. A 5 second sleep occurs, and then a new discovery message is created and
+ transmitted via the multi-cast socket to all members of the group.
+
+
+
+## Using
+
+Wrapping each of the C++ code sections above in a single file called
+*discovery.cpp* will let us make a simple build for us to test.
+
+
+### Build
+
+We can compile this with the following simple command:
+
+`g++ -std=c++0x -o test discovery.cpp NodeDiscovery.cpp
+ -I. -I/usr/include/cxx-lib/inc -lcxxasn1 -pthread -lrt`
+
+That will generate the *test* binary, which can be executed on our host
+machine. It is important not to run this in production - after all it is meant
+as an illustrative example.
+
+
+### Executing
+
+Using two systems on the same network, you should be able to insert a print
+routine that will walk the database and dump it's contents every so often.
+This should show both machines communicating. Other enhancements to add would
+be service registration and querying.
+
+Using a simple network sniffing tool, such as *tcpdump*, we can see the traffic
+going on the multicast socket:
+
+ tcpdump: listening on any, link-type LINUX_SLL (Linux cooked), capture size 262144 bytes
+ 10:46:47.868779 IP (tos 0x0, ttl 1, id 26815, offset 0, flags [DF], proto UDP (17), length 45)
+ 192.168.0.135.rwhois > 224.0.0.1.rwhois: UDP, length 17
+ 0x0000: 4500 002d 68bf 4000 0111 6fd0 c0a8 0087
+ 0x0010: e000 0001 10e1 10e1 0019 5910 0081 0101
+ 0x0020: 0203 a005 a007 3009 a00d 300f a0
+
+Notice that after the last octet of the ipv4 header (`e000 0001`) comes the
+ASN.1 encoded BER message.
+
+
+### Future
+
+As it stands, the discovery mechanism here could be greatly improved. It could,
+for instance, use a unix domain socket with a qemu VM to expose discovery via
+a non-IP network, using the same socket monitor. This could be used to perform
+things like VM orchestration, or monitoring. Additional front-ends could be
+devised, such as SOAP interfaces using the XML encoding rules. This is meant
+to be just a simple taste of what can be done.
diff --git a/doc/developer-guide.md b/doc/developer-guide.md
index 7462c28..edbedd5 100644
--- a/doc/developer-guide.md
+++ b/doc/developer-guide.md
@@ -175,3 +175,4 @@ case SNACC::HeartbeatResponses::heartbeatReplyProcessesCid:
The whole application will get compiled and linked with `-lcxxasn1` and will then
be ready to execute.
+A more complete C++ tutorial can be found [here](developer-guide-sample-app.md).
--
2.7.4
More information about the dev
mailing list