This project is a companion to the Reviewing IBM Cloud services, resources and usage tutorial. It implements an API-driven approach to obtain and visualize IBM Cloud usage and billing data.

The samples are supported by the following IBM Cloud Functions packages:

• OpenWhisk Cognos Dashboard
• OpenWhisk JSONetl
• OpenWhisk SQL Query
• Cloud Object Storage

1. Ensure that you have the appropriate roles to view billing and usage data. For instuctions, see the Assign permissions section of the tutorial.

2. Clone or download this repository.

3. Download and install wskdeploy. Add the wskdeploy executable to your PATH. It will be used to deploy the various artifacts to IBM Cloud Functions.

To deploy the application, use the below commands and installation scripts.

1. Using your terminal, change directory to the downloaded repo.

2. Login to IBM Cloud and target your Cloud Foundry account. See CLI Getting Started. Federated users should add the --sso flag.

   ibmcloud login [--sso]

   ibmcloud target --cf

3. The following commands create the needed services: Cloud Object Storage, SQL Query and Cognos Dashboard Embedded. If you already have a service deployed, choose the paid command. To change the region where these services are deployed, edit the makefile and update the REGION variable.

   1. Create Cognos Dashboard Embedded using either of the following.

      make create-cognos-lite

      make create-cognos-paid

   2. Create Cloud Object Storage using either of the following.

      make create-cos-lite

      make create-cos-paid

   3. Create SQL Query using either of the following.

      make create-sql-lite

      make create-sql-paid

4. In your browser, access your IBM Cloud Dashboard. There should be three new services that begin with usage-tutorial. Access the usage-tutorial-cos service instance.

5. Create a new bucket to store usage data.

   • Click the Create a bucket button.
   • Select Cross Region from the Resiliency drop down.
   • Select a location from the Location.
   • Provide a bucket Name and click Create If you receive an AccessDenied error, try with a more unique bucket name.

6. Set the location and bucket name as an environment variable to be used later. Examples are us-geo and ibmcloud-usage.

   export LOCATION=<your location>

   export BUCKET=<your bucket name>

7. Back in the Dashboard, access the usage-tutorial-sql instance.

8. Click the Instance CRN button (lower right corner of page) to copy it to the clipboard, and again set an environment variable.

   export INSTANCE_CRN=<your instance crn>

9. Set an environment variable to capture usage data for a given month. The format is YYYY-MM.

   export MONTH=2018-09

10. Run the following commands to create a Platform API Key and Authentication Tokens to be used with the application. The API Key will be used with SQL Query. Authentication tokens will be used to make requests to obtain billing and usage data. Run the last command to confirm all have been set.

    export API_KEY=`ibmcloud iam api-key-create usage-tutorial-key -d 'apiKey created for http://github.com/IBM-Cloud/cloud-usage-samples' | grep 'API Key' | awk ' {print $3} '`

    export IAM_TOKEN=`ibmcloud iam oauth-tokens | head -n 1 | awk ' {print $4} '`

    export UAA_TOKEN=`ibmcloud iam oauth-tokens | tail -n 1 | awk ' {print $4} '`

    echo -e API_KEY ' \t ' $API_KEY && echo -e IAM_TOKEN ' \t ' $IAM_TOKEN && echo -e UAA_TOKEN ' \t ' $UAA_TOKEN

11. Deploy the tutorial package to IBM Cloud Functions. The first command will ensure wskdeploy is properly initialized.

    ibmcloud fn package list

    make deploy

12. Bind your Cloud Object Storage and Cognos Dashboard Embedded service credentials to the packages.

    make bind-services

To obtain and process billing and usage data, you'll execute several IBM Cloud Functions sequences.

1. Obtain an account GUID to fetch usage data. The account GUID used should match the Account: value seen in the target output. Copy the account GUID to the clipboard.

   ibmcloud target && ibmcloud iam accounts

2. Start the sequence to retrieve and process resource group's usage for the account. You can watch progress using the Monitor dashboard in the IBM Cloud Functions service. The Actitivity Log should list actions with a green checkmark.

   ibmcloud fn action invoke tutorial-etl-process/resource-groups-billing -r --param guid <your guid>

3. After all data has been collected, run the sequence to collect Cloud Foundry usage for the account.

   ibmcloud fn action invoke tutorial-etl-process/cf-orgs-billing -r

4. Check your Cloud Object Storage bucket, it should now contain files that contain usage data for the various services and resources.

5. Access the usage-tutorial-sql services from the Dashboard and click the Open UI button.

6. From the command line, run the following sequence. The SQL Query UI will show progress and the result data when complete.

   ibmcloud fn action invoke tutorial-etl-query/billing-job -r

7. Once the SQL Query job has completed, return to the command line and copy the job_id returned from the sequence.

8. Run the following command to get the URL to the billing dashboard. Append the job_id value to the URL similar to this example https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/<your gateway id>/tutorial/billing?job_id=<your job id>.

   ibmcloud fn api list | grep 'Tutorial Dashboard'

9. Visit the dashboard URL in your browser to see a pre-created Cognos Embedded Dashboard with your usage data.

It's likely that the authentication tokens (UAA and IAM) have expired. Run the following to update the tutorial-etl-request package with more recent tokens.

export IAM_TOKEN=`ibmcloud iam oauth-tokens | head -n 1 | awk ' {print $4} '`

export UAA_TOKEN=`ibmcloud iam oauth-tokens | tail -n 1 | awk ' {print $4} '`

make update-request

Run the following make commands to remove the packages and services.

make undeploy

make remove-services

The following indicates that your .wskprops file is stale.

Error: servicedeployer.go [1660]: [ERROR_WHISK_CLIENT_ERROR]: Error code: 246: API creation failure: Unable to obtain API(s) from the API Gateway (status code 400): Invalid access token (expired)

Delete your /Users/<username>/.wskprops file. Then issue the following command to recreate it.

ibmcloud fn package list

Edit the makefile's bind-services target. Change the --instance flag to reflect the names of your existing services.