This article discusses about using AWS in golang using the goamz library. The official sdk can be found here; bear in mind that it is NOT used for this tutorial since it was in beta at the time of writing.

This is a fairly light-weight, beginner level tutorial to get you up and running with AWS in golang. Though using a third party library, we believe the basics could be translated to the official one.

We will attempt to provide a starting point for using Amazon Simple Queue Service (SQS) and Amazon DynamoDB. Retry,silent failure, error handling mechanisms have not been discussed in this tutorial, but from our experience can be easily added.

The following components are necessary for this tutorial:

1. AWS account
2. golang.
3. goamz
4. Terminal

The optional components are:

1. Sublime Text Editor
2. Package Control
3. GoSublime

SQS

Queue Setup

Let’s begin with the queue setup;
1- Creating a new queue using AWS console

2- Import the library;

import (  
    "github.com/goamz/goamz/sqs"
)

3- Storing required AWS credentials;

var access_key  = // Your AWS access key  
var secret_key  = // Your AWS secret key  
var queue_url   = // URL found under details on AWS console  

4- The region name is located on the top right of the AWS console;

// Region in which queue exists
var region_name = us.west.2 // For Oregon  

More information can be found here.

5- Your queue name;

var queue_name = //queue_name  

6- Establishing the connection to your AWS account in a go program;

func establish_connection(access_key string, secret_key string, region_name string) *sqs.SQS {

    // Creating a new sqs client.
    conn, cerr := sqs.NewFrom(access_key, secret_key, region_name)  
    return conn
}

The function returns a pointer to a queue object.

7- Putting it all together into queueInitialize.go;

package main

import (  
    "fmt"
    "github.com/goamz/goamz/sqs"
)

func queue_init() sqs.Queue {

    var access_key  = // Your AWS access key 
    var secret_key  = // Your AWS secret key
    var queue_url   = // URL found under details on AWS console

    // Region in which queue exists
    var region_name = us.west.2 // For Oregon

    var queue_name = //Your queue name

    conn := establish_connection(access_key, secret_key, region_name)

    q, _ := conn.GetQueue(queue_name)


    fmt.Println("queue does exist")
    q := conn.QueueFromArn(queue_url)

    return *q
}

   func establish_connection(access_key string, secret_key   string, region_name string) *sqs.SQS {

        // Creating a new sqs client.
        conn, cerr := sqs.NewFrom(access_key, secret_key, region_name)  
        return conn
}

 

Queue Interface

After the queue is setup and ready, we need to use it. This section discusses the implementation of the basic queue operations such as sending, receiving and deleting messages. To know more about SQS please read, the developer guide.

1- Import the library;

import (  
    "github.com/goamz/goamz/sqs"
)

2- Sending a string message to the queue;

//send msg
func send_msg(msg string, q *sqs.Queue) sqs.SendMessageResponse {  
    res, _ := q.SendMessage(msg)

    return *res
}

3- Receiving a single message;

//recieve msg
func get_msg(q *sqs.Queue) sqs.ReceiveMessageResponse {  
    recieveMessageResponse, err := q.ReceiveMessage(1)

    return *recieveMessageResponse
}

The response will be used at a later point in time.

4- Deleting a message;

//delete msg
func delete_msg(receiveMessageResposnse *sqs.ReceiveMessageResponse, q *sqs.Queue) sqs.DeleteMessageResponse {


deleteMessageResposne, err := q.DeleteMessageUsingReceiptHandle(receiveMessageResposnse.Messages[0].ReceiptHandle)

    return *deleteMessageResposne
}

The function uses the ReceiveMessage response returned after receiving a message from the queue. The ReceiveMessageResponse has an attribute called a ReceiptHandle for each message which can be used to delete that message.

5- Putting it all together in queueInterface.go;

import (  
        "github.com/goamz/goamz/sqs"
    )

    //send msg
    func send_msg(msg string, q *sqs.Queue) sqs.SendMessageResponse {
        res, _ := q.SendMessage(msg)

        return *res
    }

    //recieve msg
    func get_msg(q *sqs.Queue) sqs.ReceiveMessageResponse {
        recieveMessageResponse, err := q.ReceiveMessage(1)

        return *recieveMessageResponse
    }

    //delete msg
    func delete_msg(receiveMessageResposnse *sqs.ReceiveMessageResponse, q *sqs.Queue) sqs.DeleteMessageResponse {


    deleteMessageResposne, err := q.DeleteMessageUsingReceiptHandle(receiveMessageResposnse.Messages[0].ReceiptHandle)

        return *deleteMessageResposne
    }

 

Testing

1- The main function;

func main() {  
    q := queue_init()
    fmt.Println(q)

    send_msg_response := send_msg("hello", &q)
    fmt.Println(send_msg_response)

    receive_msg_response := get_msg(&q)
    fmt.Println(receive_msg_response.Messages[0].Body)

    delete_msg_response := delete_msg(&receive_msg_response, &q)
    fmt.Println(delete_msg_response)
}

2- Traverse into the folder where the program files are located and run this command in a terminal:

go run queueInitialize.go queueInterface.go  

3- The output will be of the form;

{9aa9999a9999 https://sqs.us-west-2.amazonaws.com/999999999999/my_queue_name}
{9a99999aaa9a9a99a9999a999999a999  aa99aaa9-a9a9-99aa-a99a-99a99aa999a9 {99999aa9-a999-99aa-9a99-99aaa9999aa9 9}}
hello  
{{99999aaa-a9aa-9999-99aa-99a9a9aaa99a 9}}

 

DynamoDB

To know more about DynamoDB please read the developer guide.

Database Setup

1- Creating a table using AWS console.

2- Import the library;

import (  
    "github.com/goamz/goamz/aws"
    "github.com/goamz/goamz/dynamodb"
)

3- Initializing connection;

func database_init() *dynamodb.Server {

    auth := aws_auth()

    region := aws.USWest2

    ddbs := NewFrom(auth, region)

    return ddbs
}

func NewFrom(auth aws.Auth, region aws.Region) *dynamodb.Server {  
    return &dynamodb.Server{auth, region}
}

func aws_auth() aws.Auth {  
    auth, err := aws.EnvAuth()
    return auth
}

For SQS, the connection was made manually within the program where the credentials are stored. In the above example, the credentials are stored as environment variables and the function aws.EnvAuth() is used to establish a connection. For storing and accessing environment variables in golang please read the documentation on the os package along with this example.

Putting it all together in databaseInitialize.go;

import (  
        "github.com/goamz/goamz/aws"
        "github.com/goamz/goamz/dynamodb"
    )

func database_init() *dynamodb.Server {

        auth := aws_auth()

        region := aws.USWest2

        ddbs := NewFrom(auth, region)

        return ddbs
    }

    func NewFrom(auth aws.Auth, region aws.Region) *dynamodb.Server {
        return &dynamodb.Server{auth, region}
    }

    func aws_auth() aws.Auth {
        auth, err := aws.EnvAuth()
        return auth
    }

Database Interface

Import the library;

import (  
    "github.com/goamz/goamz/dynamodb"
)

Before we get into the implementation of the database operations; let’s take a look at the helper functions which are also useful for other tasks beyond the interfacing.
2.1. Table Descriptor

// Get table description
func get_table_description(table *dynamodb.Table) *dynamodb.TableDescriptionT {  
table_description, td_err := table.DescribeTable()

return table_description  
}

2.2. DynamoDB Table

// Get a table object to perform table.Operation(param1, ... param n)
func get_table(table_name string, ddbs *dynamodb.Server) *dynamodb.Table {  
    table_descriptor, td_err := ddbs.DescribeTable(table_name)

    primary_key := build_primary_key(table_descriptor)

    table := ddbs.NewTable(table_name, primary_key)

    return table
}

2.3. DynamoDB Attributes For other non key attributes in a tuple.

// Create Dynamo DB Attribute list
func attribute_list_creator(attribute string) []dynamodb.Attribute {  
    var attribute_list = make([]dynamodb.Attribute, 1)

    // cannot have empty attribute list
    attribute := &dynamodb.Attribute{
        Type:      "S",
        Name:      "attribute_name",
        Value:     attribute,
        SetValues: make([]string, 0),
        Exists:    ""}

    attribute_list[0] = *attribute

    return attribute_list
}

2.4. DynamoDB Key

// Create Dynamo DB Key
func key_creator(primary_hash_key string) *dynamodb.Key {  
    key := &dynamodb.Key{
        HashKey: msid}

    return key
}

Writing to the database;

// Write to database.
func database_writer(primary_hash_key string, primary_range_key string, attribute_list []dynamodb.Attribute, table dynamodb.Table) bool {  
    bool, _ := table.PutItem(primary_hash_key, primary_range_key, attribute_list) // Overwrites

    return bool
}

The get_table function returns the table object used for performing operations.

Reading from the database;

// Read from Database (GetItem(key(primary_hash_key,primary_range_key)))
// Can be used for checking if record exists or not by the length of the attribute_list
// Consistent always returns the last updated value
func database_reader(table *dynamodb.Table, key *dynamodb.Key) (map[string]*dynamodb.Attribute, int) {

    attribute_map, _ := table.GetItemConsistent(key, true)

    return attribute_map, len(attribute_map)
}

The key_creator function is used to generate a key *dynamodb.Key for querying on the table to retrieve the record(s).

Deleting from the database has been left as an exercise for the reader.