AWS Bedrock Beyond the Base Models, Considering Custom Models.

I have been experimenting with AWS Bedrock for a few weeks! With more of a focus on text or content generation, and summarization, and much less on Chat or Q&A and illustration generation.

There are already many articles, blogs, and videos that cover the basics of the bedrock services, and making API/Python boto3 calls, so I won't cover those areas here.

I want to share some code examples of using Python boto3 to make calls using the base foundational models in AWS bedrock, but also using custom models that were trained on the base foundation models.

There are a couple of parameters in the code below that can be modified for your specific use case or just leave them as is and experiment with them and observe the changes in the outputs. You can change the inputText (prompt) to whatever you want. Save this code and run it as Python3 FileName.py

import boto3
import json

bedrock = boto3.client(service_name= 'bedrock-runtime', region_name= 'us-east-1')
modelId = '**amazon.titan-text-lite-v1**'
accept = 'application/json'
contentType = 'application/json'
body = json.dumps({
  "inputText": "Write a blog about EVs and the environment.",
  "textGenerationConfig": {
        "maxTokenCount": 1024,
        "temperature":0.4,
        "topP":1
        }
})

response = bedrock.invoke_model(body=body, modelId=modelId, accept=accept, contentType=contentType)    

response_body = json.loads(response.get('body').read())
outputText = response_body.get('results')[0].get('outputText')

text = outputText[outputText.index('\n')+1:]
content = text.strip()
print(outputText)

Below we can see a small change which is where we update the ModelId to be the custom-model/provisioned model throughput represented as the ARN.

import boto3
import json

bedrock_runtime = boto3.client(service_name= 'bedrock-runtime', region_name= 'us-east-1')

modelId = '**provisioned model throughputs ARN**'
accept = 'application/json'
contentType = 'application/json'
body = json.dumps({
  "inputText": "Write a blog about EVs and the environment.",
  "textGenerationConfig": {
        "maxTokenCount": 1024,
        "temperature":0.4,
        "topP":1
        }
})

response = bedrock_runtime.invoke_model(body=body, modelId=modelId, accept=accept, contentType=contentType)

response_body = json.loads(response.get('body').read())
outputText = response_body.get('results')[0].get('outputText')

text = outputText[outputText.index('\n')+1:]
content = text.strip()
print(outputText)

Two more tips that were super helpful to me!

1. If you are pre-training data based on a base model, it is critical that you save the file in the JSONL file format, and that there are no extra spaces at the bottom of the file. More info here on that- https://docs.aws.amazon.com/bedrock/latest/userguide/model-customization-prepare.html

This is an example, where we see nothing after or under the last line, otherwise, you will have parse errors when you try and fine-tune the custom model.

{"prompt": "<prompt text>", "completion": "<expected generated text>"}
{"prompt": "<prompt text>", "completion": "<expected generated text>"}
{"prompt": "<prompt text>", "completion": "<expected generated text>"}      

1. I wanted to create or generate a couple dozen lines to be used with fine-tuning similar to what's above. With domains/topics, I am familiar with this is an easier task, but for others, it can be hard to get started.

There is probably another way to tackle this, but I ended up creating a GPT [(https://chat.openai.com/g/g-2qfCPili0-dataset-trainer)] (you will need ChatGPT Plus to access it), Maybe something I can do with AWS PartyRock and PartySmith which worked well for this ideation/creation step.

With the GPT mentioned above:

1. Just drop in a PDF or whatever data/file you are working with.
2. Is this for pre-training or fine-tuning?
3. Ask the GPT What completion text suits this prompt for fine-tuning.
4. Now the fun part, we ask to generate some pre-training text that's in JSONL format, and how many lines you want, in my case I asked for 25-30.

The JSONL format is whats uploaded to the AWS S3 bucket and used for fine-tuning or the continued pre-training for example.