Call Vertex AI LLM from your ABAP environment with ABAP SDK for Google Cloud

Call Vertex AI LLM from your ABAP environment with ABAP SDK for Google Cloud

About this codelab

subjectLast updated Feb 28, 2024
account_circleWritten by Devesh Singh

1. Introduction

In this codelab, we have listed the steps on how to call the Google Cloud Vertex AI's PaLM 2 Text (text-bison) foundation model from your ABAP environment using ABAP SDK for Google Cloud.

The following Google Cloud services are used in this codelab:

  • Vertex AI
  • Cloud Shell

What you'll build

You'll create the following:

  • Enable Vertex AI API in a Google Cloud project.
  • Pass a prompt to the Vertex AI's PaLM 2 Text model and receive response using ABAP SDK for Google Cloud.

2. Requirements

  • A browser, such as Chrome or Firefox.
  • A Google Cloud project with billing enabled or Create a 90-Day Free Trial account for Google Cloud Platform.
  • SAP GUI (Windows or Java) installed in your system. If SAP GUI is already installed on your laptop, connect to SAP using the VM external IP address as the Application Server IP. If you are on Mac, then you can also install the SAP GUI for Java available in this link.

3. Before you begin

6757b2fb50ddcc2d.png

  • Run the following commands to authenticate for your account and set the default project to abap-sdk-poc. Zone us-west4-b is used as an example. If needed, please change the project and zone in the following commands based on your preference.
gcloud auth login
gcloud config
set project abap-sdk-poc
gcloud config
set compute/zone us-west4-b

4. Overview

Here is a quick overview of some of the entities of Vertex AI that you would be working with in this codelab:

  • PaLM 2 for Text: The PaLM 2 for Text (text-bison, text-unicorn) foundation models are optimized for a variety of natural language tasks such as sentiment analysis, entity extraction, and content creation.
  • Text-bison: Text-bison is a large language model (LLM) developed by Google AI. It's a foundational model for GenAI that can understand and generate language.

5. Enable Vertex AI in your Google Cloud Project

  1. Go to your Google Cloud Project and Click on Activate Cloud Shell in the top right corner.

8d15f753321c53e6.png

  1. Once the shell is activated, execute the below command to enable the Cloud Secret Manager API in your Google Cloud Project.
gcloud services enable aiplatform.googleapis.com

You should now have the Vertex AI API enabled in your Google Cloud Project.

6. Create a Service Account with user roles

To create a service account with required roles, perform the following steps:

  1. Run the following command in the Cloud Shell terminal:
gcloud iam service-accounts create abap-sdk-vertexai-tester \
--display-name="Service Account for Vertex AI"
  1. Now add the required role to the service account created in the above step to access the API.
gcloud endpoints services add-iam-policy-binding aiplatform.googleapis.com \ --member='serviceAccount:abap-sdk-vertexai-tester@abap-sdk-poc.iam.gserviceaccount.com' \
--role='roles/roles/aiplatform.endpoints.predict'

The above commands use abap-sdk-poc as template name for the Google Cloud Project. Replace it with your project id.

  1. To verify, the role has been added, go to IAM page. The service account you created should be listed along with the role that has been assigned to it.

7. Create SDK configurations in SAP

Now that you have set up the pre-requisites on the Google Cloud side, we can move ahead with the configuration on the SAP side. For authentication and connectivity related configuration, the ABAP SDK for Google Cloud uses tables /GOOG/CLIENT_KEY and /GOOG/SERVIC_MAP.

To maintain the configuration in the table /GOOG/CLIENT_KEY table, perform the following steps:

  1. In the SAP GUI, enter transaction code SPRO.
  2. Click SAP Reference IMG.
  3. Click ABAP SDK for Google Cloud > Basic Settings > Configure Client Key.
  4. Maintain the following values against the fields:

Field

Description

Google Cloud Key Name

VERTEX_AI_DEMO

Google Cloud Service Account Name

abap-sdk-vertexai-tester@abap-sdk-poc.iam.gserviceaccount.com

Google Cloud Scope

https://www.googleapis.com/auth/cloud-platform

Google Cloud Project Identifier

abap-sdk-poc

Authorization Class

/GOOG/CL_AUTH_GOOGLE

Leave the other fields blank

Vertex AI has region specific endpoints, we would have to create a RFC destination for the endpoint and build the mapping for the API in table /GOOG/SERVIC_MAP against the client key. (Refer to this link for region specific service endpoints for Vertex AI).

Below are the steps to create a RFC for the codelab.

  • Go to transaction code SM59 and create a type "G" connection.
  • Give the name of the destination as "ZGOOG_VERTEXAI_V1".
  • Give the descriptions as required and port as "443".
  • Give the "Host" address as " us-west4-aiplatform.googleapis.com" as we are referring to location "us-west4-b" in the codelab.

Below is a screenshot of the RFC destination for your reference.

54316c5de58624c7.png

To maintain the configuration in the table /GOOG/SERVIC_MAP table, perform the following steps:

  1. In the SAP GUI, enter transaction code SPRO.
  2. Click SAP Reference IMG.
  3. Click ABAP SDK for Google Cloud > Basic Settings > Configure Service Mapping.
  4. Maintain the following values against the fields:

Field

Description

Google Cloud Key Name

VERTEX_AI_DEMO

Google Service Name

aiplatform:v1

RFC Destination

ZGOOG_VERTEXAI_V1

8. Invoke PaLM 2 Text using the SDK

  1. Log in to your SAP system.
  2. Go to the transaction code SE38 and create a report with the name "ZDEMO_VERTEX_AI".
  3. In the pop-up that opens up, provide details as shown below and click Save.

b06fda1a879290f7.png

In the next pop-up, either select Local Object or provide a package name as appropriate.

  1. In the ABAP Editor, add the following code snippet:
  • In this snippet, as a prompt, we are passing the email content for an urgent order request along with instructions to parse it to extract order attributes such as Customer Name, Company Name, Customer Designation, Item Name, Order Quantity, Shipping address, and Delivery Date.
  • The ABAP types declarations are to prepare the request for and capture the response from Vertex AI PaLM 2 Text model as per the documentation here.
* Types declarations
TYPES:
  BEGIN OF ty_instances,
    content TYPE string,
  END OF ty_instances.

TYPES:
     tt_instances TYPE STANDARD TABLE OF ty_instances WITH DEFAULT KEY .

TYPES:
  BEGIN OF ty_parameters,
    max_output_tokens TYPE i,
    temperature       TYPE f,
    top_k             TYPE i,
    top_p             TYPE f,
  END OF ty_parameters.

TYPES ty_categories TYPE string .
TYPES:
  BEGIN OF ty_scores,
    scores TYPE string,
  END OF ty_scores .
TYPES:
  tt_categories TYPE STANDARD TABLE OF ty_categories WITH DEFAULT KEY .
TYPES:
  tt_scores TYPE STANDARD TABLE OF ty_scores WITH DEFAULT KEY .

TYPES:
  BEGIN OF ty_safety_attributes,
    blocked    TYPE abap_bool,
    categories TYPE tt_categories,
    scores     TYPE tt_scores,
  END OF ty_safety_attributes .

TYPES:
  BEGIN OF ty_predictions,
    content           TYPE string,
    safety_attributes TYPE ty_safety_attributes,
  END OF ty_predictions .
TYPES:
  tt_predictions TYPE STANDARD TABLE OF ty_predictions WITH DEFAULT KEY .

TYPES:
  BEGIN OF ty_output,
    deployed_model_id  TYPE string,
    metadata           TYPE REF TO data,
    model              TYPE string,
    model_display_name TYPE string,
    model_version_id   TYPE string,
    predictions        TYPE tt_predictions,
  END OF ty_output.

* Data declarations
DATA:
  lv_p_projects_id   TYPE string,
  lv_p_locations_id  TYPE string,
  lv_p_publishers_id TYPE string,
  lv_p_models_id     TYPE string,
  ls_input           TYPE /goog/cl_aiplatform_v1=>ty_001.

DATA:
      lv_email_text TYPE string.

CONSTANTS: lc_ob TYPE c VALUE '{',
           lc_cb TYPE c VALUE '}'.

* Email content having order request
lv_email_text = |'Hello, We are running low on Dark Chocolates, please ship 20 units of "Lindt 90% Dark Chocolates"'| &&
                |' and 10 units of "Toblerone Dark Bar 100g" to "Google Cloud, 1 East 2'| &&
                |'4th St, Kearney NE" by as early as possible. Thanks and Regards, John Doe, Procurement Manager, Google Cloud'|.

TRY.
    DATA(lv_raw) = VALUE string( ).
* Open HTTP Connection
    DATA(lo_client) = NEW /goog/cl_aiplatform_v1( iv_key_name = 'VERTEX_AI_DEMO' ).

* Populate relevant parameters
    lv_p_projects_id = lo_client->gv_project_id.
    lv_p_locations_id = 'us-west4-b'.
    lv_p_publishers_id = 'google'.
    lv_p_models_id = 'text-bison'.

* Call API method: aiplatform.projects.locations.publishers.models.predict
    CALL METHOD lo_client->predict_models
      EXPORTING
        iv_p_projects_id   = lv_p_projects_id
        iv_p_locations_id  = lv_p_locations_id
        iv_p_publishers_id = lv_p_publishers_id
        iv_p_models_id     = lv_p_models_id
        is_input           = VALUE #(
     parameters = NEW ty_parameters(
           max_output_tokens  = 256
           temperature = '0.2'
           top_k = '40'
           top_p  = '0.8' )
* Pass the prompt with email content and instructions to get order attributes
     instances = NEW tt_instances( ( content =
      |'Give me the Customer Name, Company Name, Customer Designation, Item Name, Order Quantity, Shipping address, Delivery Date'| &&
      |'from an email context. Structure the response in JSON camelcase format with fields corresponding to each entities. Place the ordered items in a JSON nest'| &&
      |'Default the delivery date to a future date in DD.MM.YYYY format. Here is the email content:'| && lv_email_text ) ) )
      IMPORTING
        es_raw             = lv_raw
        ev_ret_code        = DATA(lv_ret_code)
        ev_err_text        = DATA(lv_err_text)
        es_err_resp        = DATA(ls_err_resp).

    IF lo_client->is_success( lv_ret_code ).
      DATA(ls_output_llm) = VALUE ty_output( ).
* Deserializing API response to get model response
      /goog/cl_json_util=>deserialize_json( EXPORTING iv_json        = lv_raw
                                                      iv_pretty_name = /ui2/cl_json=>pretty_mode-extended
                                            IMPORTING es_data        = ls_output_llm ).
      cl_demo_output=>new(
        )->begin_section( 'API Call Successful:'
        )->write_text( 'Respose from model:'
        )->write_text( ls_output_llm-predictions[ 1 ]-content
        )->display( ).
    ELSE.
      cl_demo_output=>new(
        )->begin_section( 'API Call Unsuccessful:'
        )->write_text( 'Error Message:'
        )->write_text( lv_err_text
        )->display( ).

    ENDIF.

* Close HTTP Connection
    lo_client->close( ).

  CATCH /goog/cx_sdk INTO DATA(lo_exception).
    DATA(lv_error) = lo_exception->get_text( ).
    cl_demo_output=>new(
     )->begin_section( 'Exception Occured:'
     )->write_text( lv_error
     )->display( ).

ENDTRY.
  1. Save and activate the report program.
  2. Execute the report (F8).

On successful execution you should see the report output as shown below with the extracted order attributes as instructed in the prompt.

c40839b43b8ac44f.png

9. Congratulations

Congratulations! You have successfully called Vertex AI PaLM 2 Text model from your ABAP program to parse an order request using ABAP SDK for Google Cloud.

Instead of hard coding, you can also think of getting the email content from your Gmail inbox using ABAP SDK for Google Cloud, and then passing the content to Vertex AI as described in this codelab.

10. Clean up

If you do not wish to continue with the additional codelabs related to ABAP SDK for Google Cloud, please proceed with the cleanup.

Delete the project

  • Delete the Google Cloud project:
gcloud projects delete abap-sdk-poc

Delete individual resources

  1. Delete the compute instance:
gcloud compute instances delete abap-trial-docker
  1. Delete the firewall-rules:
gcloud compute firewall-rules delete sapmachine
  1. Delete the service account:
gcloud iam service-accounts delete \
    abap-sdk-dev@abap-sdk-poc.iam.gserviceaccount.com