DEV Community

Agustin Perez Paladini
Agustin Perez Paladini

Posted on

packing / unpacking data in c++

Hi there, newbie here, and this is my first post. I decided to restructure some of my github repos since they maybe can be useful to someone :).

The post:

Packet lib

This is a very short blog post since its a small c++ header-only lib I created, hence the full documentation would be in the github repo.
The packet library is intended to provide an easy "API" for packing and unpacking data (messages) so they can, for example, be safely sent over the wire (TCP). Whenever you need to send information over the wire is important to know:

  • They are respecting some kind of protocol (otherwise you may want to close the connection).
  • What you are getting is complete (need to know when the message start / end).

TCP ensures you that the data will be sent in order, but doesn't ensure that the calls are atomic (reads / writes). Hence, whatever you the data you are sending, needs to be "packed" / "unpacked". For example, protocol buffers is a library that serializes messages, but doesn't handle where the message start / end.

How it works

The packets have the following shape:
[ head_pattern | pkt_content_len | content | tail_pattern ]

  • head_pattern: (optional) a user defined pattern to detect early wrong or invalid messages over the wire
  • pkt_content_len: field indicating the size of the content buffer
  • content: the data / content itself
  • tail_pattern: (optional) ensure that the packet size was correct and respects the protocol.

The intention of the packet library, is to provide an API that let the user read as few characters as needed while checking the validity of the packet while reading it.

Feedback

As always, very happy to get some feedback, comments, or join forces on the open community.

Salute

Top comments (5)

Collapse
 
pgradot profile image
Pierre Gradot • Edited

Why not using websocket for this purpose? I remember using it to send JSON object a few years ago. Websocket handles the begin/end of a "packet" over a TCP stream.

For what I undertstand, you library is more generic because I can be used with something else that TCP.

Collapse
 
agudpp profile image
Agustin Perez Paladini

Hi Pierrre, that may work, this is more general purpose, is data format agnostic, I used this on limited iot projects where the messages being sent were binary format (in particular protocol buffers).
Thanks for the comment :)

Collapse
 
ajeetht profile image
Ajeeth thangarasu

The packet shape is as TCP packet structure is it?

Collapse
 
agudpp profile image
Agustin Perez Paladini

hi @Ajeeth. No, this is on top of TCP (application level). It can be used with any communication protocol.

Collapse
 
ajeetht profile image
Ajeeth thangarasu

Oh ok