Paradigm Shift in Software Development, Part 2

• Paradigm Shift in Software Development, Part 1
• Paradigm Shift in Software Development, Part 2

Last time, we mentioned that GenAI can be used to implement business logic and dramatically increase development productivity and reduce the effort of debugging.

However, I have to say that GenAI is not suitable for applications that require accurate computation. In other words, the demonstration in the previous article was just a demo and not a recommendation to use GenAI for calculating promotions.

But that doesn't mean using GenAI as a business logic is a bad idea. On the contrary, GenAI is really suitable for replacing "some" business logic.

Maybe you will ask, most of the business logic needs to be calculated and requires accuracy, so what exactly can GenAI help? Well, I'll tell you, it helps call the APIs that have already been coded.

The correct way to use GenAI to handle business logic is to encapsulate that business logic directly into a Remote Procedure Call (RPC) and let GenAI prepare the parameters needed for the RPC and call the corresponding RPC correctly.

Currently, the RPC that is easiest for GenAI to handle is the REST API for the following reasons.

1. Plain text style, no matter the URI, HTTP header, HTTP method, query parameter and request payload are all plain text.
2. Full specification support. Nowadays, the most straightforward way to describe a REST API is to use a swagger, and the description file of the swagger is also plain text.
3. REST is a mature and relatively less dependency choice, on the contrary, such as gRPC and other protocols need to have to install additional drivers.

Therefore it's pretty clear what we're going to do, so let's follow the steps and explain them. We will still use Gemini as a demonstration as in the previous article, but once again, you can use what you are familiar with.

Experimental environment introduction

I have prepared a web service with basic CRUD.

https://github.com/wirelessr/genai_api_calling

Just docker compose up to get the service up.

This service is a product microservice that handles CRUD for a single product and saves the results in a database. For ease of use, I'm using Redis as the database for this example.

This service has two entry points.

1. http://localhost:50000 is the home page of the microservice, with a simple list page and blocks for adding and modifying products.
2. http://localhost:50000/apidocs is the swagger home page, which lists all the API descriptions and specifications.

Then prepare some basic test data.

curl -X POST "http://localhost:50000/api/product" -H "accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d "id=123&name=Apple&description=Fruit"
curl -X POST "http://localhost:50000/api/product" -H "accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d "id=234&name=Bird&description=Animal"
curl -X POST "http://localhost:50000/api/product" -H "accept: application/json" -H "Content-Type: application/x-www-form-urlencoded" -d "id=345&name=Cat&description=Animal"

It is worth noting that creating a new product and modifying a product are actually the same API: POST /api/product. When the product ID doesn't exist, the API will create a new product; on the other hand, if the product ID exists, then it will modify the product.

Remember this, because it's relevant to the business logic we're trying to implement.

The following examples are actually listed in rest_api_calling.ipynb.

GenAI calls RPC

First, we need to enable GenAI to call RPCs, which is called function calling, and both OpenAI's ChatGPT and Google's Gemini have similar capabilities.

Reference links are listed here.

• ChatGPT function calling
• Gemini function calling

The following is an example.

def get_website_content(url: str) -> str:
  """Get the content from a specific URL similar to curl -X GET.

  Args.
      url (str, required): The target URL is either remote or local.

  Returns.
      str: The raw content from a specific URL.
  """
  response = requests.get(url)
  return response.text

def post_request(url: str, data: str) -> dict:
  """
  Send a request similar to curl -X POST.

  Args.
      url (str, required): Target URL.
      data (str, required): The data of the form in "id=value1&name=value2&description=value3" format.

  Returns.
      dict: Dictionary containing the status code and response content.
  """
  response = requests.post(url, headers={"Content-Type": "application/x-www-form-urlencoded"}, data=data)

  try:
      response_content = response.json()
  except ValueError:
      response_content = response.text

  return {
      "status_code": response.status_code,
      "content": response_content,
  }

system_instruction = '''You are a professional web crawler and familiar with swagger usage.
You can get the content you want by yourself through web api, 
and when you utilize web api, you will actively list which api is used.
If you don't know the answer to a question, just answer no, don't make up an answer.
'''

model = genai.GenerativeModel(model_name='gemini-1.5-pro',
                              tools=[get_website_content, post_request],
                              system_instruction=system_instruction,)

We have designed two tools for Gemini, get_website_content and post_request. The code comments and annotations must be written in detail so that GenAI can realize the use of these tools.

Now GenAI can use GET and POST.

GenAI understands RPC specifications

Once GenAI is able to invoke RPC, we then need to enable GenAI to learn "all" RPC specifications.

In the case of REST APIs, the simplest way is to use a swagger to describe all the APIs, including their inputs and outputs, as well as their functionality. The more detailed the information, the more GenAI can operate correctly without having to spend a lot of effort tuning prompts.

chat_session = model.start_chat(
    enable_automatic_function_calling=True,
    history=[
        {
            "role": "user",
            "parts": [
                "I have a website and this website provides full swagger: http://localhost:50000/apispec_1.json Please tell me the features of this website."
            ]
        },
        {
            "role": "model",
            "parts": [
                """This website provides four API endpoints:

- Create or update a product: You can send a POST request to `/api/product` with product ID, name, and description to create or update a product.
- Delete a product: You can send a POST request to `/api/product/delete/{product_id}` to delete a product by its ID.
- Get a product: You can send a GET request to `/api/product/{product_id}` to retrieve the details of a product by its ID.
- Get all products: You can send a GET request to `/api/products` to get a list of all products. """
            ]
        }
    ]
)

This is the record I left after talking to Gemini beforehand, it can be used as the history of the prompt.

You can also do some additional conversations to verify that Gemini has really learned it.

response = chat_session.send_message("Create a new product by your own thought")

print(response.text)

Practical Business Logic

Now we've made sure that GenAI knows the basics, we can make GenAI implement business logic based on those basics.

For example, the current POST /api/product is a combination of creation and modification, with the id determining whether to create or modify. The id is filled in by the client itself, so it is very likely to be wrong.

Then we can ask GenAI to find out the unused id to add based on the result of list.

Avoid the existing ids and create a new product.

In this way, the client does not need to fill in the id itself but GenAI is responsible for generating the id.

Or another use case, right now the description is free format, so it can be written any way we want. But we can use GenAI to provide a template so that all creations and modifications are in a fixed format, such as the following template.

When creating and modifying products, the description must conform to the following format.

• Category: str, the category of the product.
• Price: int, the price of the product.
• Notes: str, additional information.

The implementation of this business logic doesn't involve computation, it's just a matter of letting GenAI call on a known API to accomplish a specific goal.

You can use your imagination to make more variations.

Conclusion

In the previous article we mentioned that GenAI can be used to implement business logic, and this is true. But in practice, we still need to do some development work to make GenAI able to implement business logic accurately.

Because GenAI may be good at business logic, but it's not accurate, so in order to maximize GenAI's strengths, we need to make GenAI do as little computation as possible. By encapsulating the business logic that needs to be computed and enabling GenAI to execute it exactly according to the instructions we provide, we can maximize productivity.

Why do we use GenAI to implement business logic?

Let's go back and answer this fundamental question. Because GenAI has the ability to understand natural language and execute our predefined scripts or steps, implementing business logic with GenAI becomes software development in natural language.

Of course, GenAI is not perfect, there are a few core elements in the development process.

1. Integration must be done properly. Although the examples I have provided are all working scripts, in fact, to make GenAI actually put into production environment requires a lot of infrastructure. For example, LLM caching, vector databases, and model repository. Each of these components is a new tech stack for organizations that have never introduced GenAI before.
2. Prompts must be good. Although we are developing in natural language, GenAI can easily "learn the wrong way" if we are not precise enough. Moreover, GenAI may perform normally but crash when it encounters a specific pattern, which will be very difficult to debug.
3. Testing must be done right. We have already encapsulated business logic into RPC for GenAI, but we still need to make sure that GenAI works properly in all kinds of scenarios, so we must have a high coverage of test cases.

As we can see, although software development with GenAI may seem attractive, there are many challenges that must be overcome. If we want to become a master of prompting, we still have to be a developer first.