In the first installment of this article series, we discussed how to read a “big” chunk of data from the raw body of an HTTP POST method and save it to a database as a stream property of a  class. Now let’s look at how to save such data and metadata in JSON format.

Unfortunately, Advanced REST Client doesn’t let you compose JSON objects with binary data as a value of a key (or maybe I simply haven’t figured out how to do it), so I decided to write a simple client in ObjectScript to send data to the server.

I created a new class called RestTransfer.Client and added to it parameters Server = "localhost" and Port = 52773 to describe my web server. And I created a GetLink class method in which I create a new instance of class %Net.HttpRequest and set its properties to the aforementioned parameters. 

To actually send the POST request to a server, I created a class method SendFileDirect, which reads the file I wish to send to the server and writes its contents to my request’s EntityBody property. 

After this, I call method Post("/RestTransfer/file") and, if it completes successfully, the response will be in the HttpResponse property. 

To see the result returned by the server, I call the response’s OutputToDevice method. 

Here’s the class method:

Class RestTransfer.Client
{
  Parameter Server = "localhost";
  Parameter Port = 52773;
  Parameter Https = 0;

  ClassMethod GetLink() As %Net.HttpRequest
  {
      set request = ##class(%Net.HttpRequest).%New()       
      set request.Server = ..#Server
      set request.Port = ..#Port
      set request.ContentType = "application/octet-stream" 
      quit request
  }

  ClassMethod SendFileDirect(aFileName) As %Status
  {
    set sc = $$$OK 
    set request = ..GetLink() 
    set s = ##class(%Stream.FileBinary).%New()
    set s.Filename = aFileName
    While 's.AtEnd 
    {
      do request.EntityBody.Write(s.Read(.len, .sc))
      Quit:$System.Status.IsError(sc)
    }
    Quit:$System.Status.IsError(sc)

    set sc = request.Post("/RestTransfer/file")                            
    Quit:$System.Status.IsError(sc)
    set response=request.HttpResponse
    do response.OutputToDevice()
    Quit sc
  }
}

We can call this method to transfer the same files used in the previous article:

 do ##class(RestTransfer.Client).SendFileDirect("D:\Downloads \2020_1012_114732_020.JPG")
 do ##class(RestTransfer.Client).SendFileDirect("D:\Downloads\Archive.xml")
 do ##class(RestTransfer.Client).SendFileDirect("D:\Downloads\arc-setup.exe")

We’ll see the responses from the server for each file:

HTTP/1.1 200 OK
CACHE-CONTROL: no-cache
CONTENT-LENGTH: 15
CONTENT-TYPE: text/html; charset=utf-8
DATE: Thu, 05 Nov 2020 15:13:23 GMT
EXPIRES: Thu, 29 Oct 1998 17:04:19 GMT
PRAGMA: no-cache
SERVER: Apache
{"Status":"OK"}

And we’ll have the same data in globals as before:

This means our client works and we can now expand it to send JSON to the server.

Getting to Know JSON

JSON is a lightweight format for storing and transporting data where the data is in name/value pairs separated by commas. In this case, the JSON will look something like this:

{
  "Name": "test.txt",
  "File": "Hello, world!"
}

IRIS has several classes that let you work with JSON. I’m going to use the following:

• %JSON.Adaptor provides a way to serialize a JSON enabled object as a JSON document and vice versa.
• %Library.DynamicObject provides a way to dynamically assemble data that can be conveniently passed between a web client and a server.

You can find more about these and other classes that let you work with JSON in the Class Reference section of the documentation and Application Development Guides.

The main advantage of these classes is that they give you an easy way to create JSON from an IRIS object or create an object from JSON. 

I’ll show two approaches of how you can work with JSON to send/receive files, namely one that uses %Library.DynamicObject to create an object on the client side and serialize it to a JSON document and %JSON.Adaptor to deserialize it back to RestTransfer.FileDesc on the server side, and another where I manually form a string to send to and parse on the server side. To make it more readable, I’ll create two different methods on the server for each approach.

For now, let’s concentrate on the first one. 

Creating JSON with IRIS

To start, let’s inherit the RestTransfer.FileDesc class from %JSON.Adaptor. This lets us use instance method %JSONImport() to deserialize a JSON document directly into an object. Now, this class will look as follows: 

Class RestTransfer.FileDesc Extends (%Persistent, %JSON.Adaptor)
{
  Property File As %Stream.GlobalBinary;
  Property Name As %String;
}

Let’s add a new route to UrlMap in the class broker:

<Route Url="/jsons" Method="POST" Call="InsertJSONSmall"/>

This specifies that, when the service receives a POST command with URL /RestTransfer/jsons, it should call the InsertJSONSmall class method. 

This method expects to receive JSON-formatted text with two key-value pairs where the contents of the file will be less than the maximum length of a string. I’ll set the Name and File properties of the object, save it to the database, and return the status and a JSON-formatted message indicating either success or an error. 

Here’s the class method.

ClassMethod InsertJSONSmall() As %Status
{
  Set result={}
  Set st=0 

  set f = ##class(RestTransfer.FileDesc).%New()  
  if (f = $$$NULLOREF) {    
   do result.%Set("Message", "Couldn't create an instance of class") 
  } else {  
     set st = f.%JSONImport(%request.Content) 
     If $$$ISOK(st) {
        set st = f.%Save()
        If $$$ISOK(st) {      
           do result.%Set("Status","OK")        
        } else {           
           do result.%Set("Message",$system.Status.GetOneErrorText(st)) 
        }
     } else {         
        do result.%Set("Message",$system.Status.GetOneErrorText(st)) 
     }
  } 
  write result.%ToJSON()
  Quit st
}

What does this method do? It gets the Content property of the %request object and, using method %JSONImport inherited from the class %JSON.Adaptor, converts it to a RestTransfer.FileDesc object.

If there are problems during conversion, we form a JSON with the error description:

{"Message",$system.Status.GetOneErrorText(st)}

Otherwise we save the object. If it saves without problems, we create a JSON {"Status","OK"} message. If not, a JSON message with the error description. 

Finally, we write the JSON to a response and return status st.

On the client side I added a new class method SendFile to send files to the server. The code is very similar to that of SendFileDirect but, instead of writing file contents directly to the EntityBody property, I create a new instance of the class %Library.DynamicObject, set its property Name equal to the name of the file, and encode and copy the contents of the file into property File. 

To encode the contents of the file I use the method Base64EncodeStream() proposed by Vitaliy Serdtsev.

Next, using method %ToJSON of class %Library.DynamicObject, I serialize my object to a JSON document, write it into the body of the request, and call method Post("/RestTransfer/jsons"). 

Here’s the class method:

ClassMethod SendFile(aFileName) As %Status
{
  Set sc = $$$OK
  Set request = ..GetLink() 
  set s = ##class(%Stream.FileBinary).%New()
  set s.Filename = aFileName
  set p = {}
  set p.Name = s.Filename 
  set sc = ..Base64EncodeStream(s, .t)
  Quit:$System.Status.IsError(sc)

  While 't.AtEnd { 
    set p.File = p.File_t.Read(.len, .sc)
    Quit:$System.Status.IsError(sc)
  }  
  do p.%ToJSON(request.EntityBody)
  Quit:$System.Status.IsError(sc)

  set sc = request.Post("/RestTransfer/jsons")                                    
  Quit:$System.Status.IsError(sc)

  Set response=request.HttpResponse
  do response.OutputToDevice()
  Quit sc
}

We call this method and method SendFileDirect to transfer several small files:

 do ##class(RestTransfer.Client).SendFile("D:\Downloads\Outline Template.pdf")      
 do class(RestTransfer.Client).SendFileDirect("D:\Downloads\Outline Template.pdf")    
 do ##class(RestTransfer.Client).SendFile("D:\Downloads\pic3.png")         
 do ##class(RestTransfer.Client).SendFileDirect("D:\Downloads\pic3.png")         
 do ##class(RestTransfer.Client).SendFile("D:\Downloads\Archive (1).xml")      
 do ##class(RestTransfer.Client).SendFileDirect("D:\Downloads\Archive (1).xml")    

We’ll get the following results:

As you can see, the lengths are the same and, if we save those files to a hard drive, we’ll see that they’re unchanged.

Forming JSON Manually

Now let’s concentrate on the second approach, which is to form the JSON manually. To do this, let’s add a new route to UrlMap in the class broker:

<Route Url="/json" Method="POST" Call="InsertJSON"/>

This specifies that when the service receives a POST command with URL /RestTransfer/json, it should call the InsertJSON class method. In this method, I expect to receive the same JSON, but I don’t impose any limits on the length of the file. Here’s the class method:

ClassMethod InsertJSON() As %Status
{
  Set result={}
  Set st=0 

  set t = ##class(%Stream.TmpBinary).%New()
  While '%request.Content.AtEnd 
  {        
    set len = 32000
    set temp = %request.Content.Read(.len, .sc)
    set:len<32000 temp = $extract(temp,1,*-2)  
    set st = t.Write($ZCONVERT(temp, "I", "RAW"))                                                
  }
  do t.Rewind()

  set f = ##class(RestTransfer.FileDesc).%New()  
  if (f = $$$NULLOREF) 
  {     
    do result.%Set("Message", "Couldn't create an instance of class") 
  } else {
    set str = t.Read()
    set pos = $LOCATE(str,""",")
    set f.Name = $extract(str, 10, pos-1)
    do f.File.Write($extract(str, pos+11, *))
    While 't.AtEnd {       
      do f.File.Write(t.Read(.len, .sc))
    }

    If $$$ISOK(st) 
    {
      set st = f.%Save()
      If $$$ISOK(st) 
      {         
        do result.%Set("Status","OK")         
      } else {                      
        do result.%Set("Message",$system.Status.GetOneErrorText(st)) 
      }
    } else {         
       do result.%Set("Message",$system.Status.GetOneErrorText(st)) 
    }    
  }
  write result.%ToJSON()
  Quit st
}

What does this method do? First, I create a new instance of a temporary stream, %Stream.TmpBinary, and copy into it the contents of the request. 

Because I’m going to work with it as a string, I need to get rid of a trailing quote mark (") and a curly brace (}). To do this, in the last chunk of the stream I leave out the last two characters: $extract(temp,1,*-2).

At the same time, I convert my string using the "RAW" translation table: $ZCONVERT(temp, "I", "RAW").

Then I create a new instance of my class RestTransfer.FileDesc and do the same checks as in the other methods. I know the structure of my string, so I extract the name of the file and the file itself and set them to corresponding properties.

On the client side, I modified my class method SendFile and, before forming the JSON, I check the length of the file. If it’s less than the 2,000,000 bytes (apparent limit for the %JSONImport method), I call Post("/RestTransfer/jsons"). Otherwise I call Post("/RestTransfer/json"). 

Now the method looks like this:

ClassMethod SendFile(aFileName) As %Status
{
  Set sc = $$$OK
  Set request = ..GetLink()
  set s = ##class(%Stream.FileBinary).%New()
  set s.Filename = aFileName
  if s.Size > 2000000 //3641144 max length of the string in IRIS
  {   
    do request.EntityBody.Write("{""Name"":"""_s.Filename_""", ""File"":""")      
    While 's.AtEnd 
    {       
      set temp = s.Read(.len, .sc)
      do request.EntityBody.Write($ZCONVERT(temp, "O", "RAW"))   
      Quit:$System.Status.IsError(sc)
    }

    do request.EntityBody.Write("""}")  
    set sc = request.Post("/RestTransfer/json")                           
  } else {
    set p = {}
    set p.Name = s.Filename
    set sc = ..Base64EncodeStream(s, .t)
    Quit:$System.Status.IsError(sc)
    While 't.AtEnd {
        set p.File = p.File_t.Read(.len, .sc)
        Quit:$System.Status.IsError(sc)
    } 
    do p.%ToJSON(request.EntityBody)
    Quit:$System.Status.IsError(sc)
    set sc = request.Post("/RestTransfer/jsons")                                      
  }

  Quit:$System.Status.IsError(sc)
  Set response=request.HttpResponse
  do response.OutputToDevice()
  Quit sc
}

To call the second method, I create the string with JSON manually. And to transfer the file itself, on the client side I also convert the contents using the “RAW” translation table: ($ZCONVERT(temp, "O", "RAW").

Now we call this method and method SendFileDirect to transfer several files of different sizes:

 do ##class(RestTransfer.Client).SendFile(“D:\Downloads\pic3.png”)         
 do ##class(RestTransfer.Client).SendFileDirect(“D:\Downloads\pic3.png”)         
 do ##class(RestTransfer.Client).SendFile(“D:\Downloads\Archive (1).xml”)      
 do ##class(RestTransfer.Client).SendFileDirect(“D:\Downloads\Archive (1).xml”)    
 do ##class(RestTransfer.Client).SendFile(“D:\Downloads\Imagine Dragons-Thunder.mp3”)         
 do ##class(RestTransfer.Client).SendFileDirect(“D:\Downloads\Imagine Dragons-Thunder.mp3”)
 do ##class(RestTransfer.Client).SendFile(“D:\Downloads\ffmpeg-win-2.2.2.exe”)   
 do ##class(RestTransfer.Client).SendFileDirect(“D:\Downloads\ffmpeg-win-2.2.2.exe”) 

We’ll get the following results:

We can see that the lengths are the same so everything works as it’s supposed to.

Wrapping Up

Again, you can read more about creating REST services in the documentation. The example code for both approaches is on GitHub and InterSystems Open Exchange. 

As for transferring results of the TWAIN module discussed in the first article of this series, it depends on the size of data you’ll be receiving. If the maximum resolution will be limited and the files will be small, you can use system classes %JSON.Adaptor and %Library.DynamicObject. But to be on the safe side, it’s better to either send a file directly in the body of a POST command or to form the JSON manually. Also it may be a good idea to use range requests and divide big files into several parts, send them separately and consolidate back into one file on the server side.

In any case, if you have any questions or suggestions, please don’t hesitate to write them in the comments section.