Check Azure CosmosDB “currency” from code and adapt the throughput

Azure CosmosDB is a scalable, distributed, highly responsive and cloud based database service. It boasts with low latency and pricing based on the usage.

You are billed on hourly basis by provisioning total Request units (RU) per second for Azure CosmosDB database or collection. RUs can be described as a rate-based currency for throughput presented in RUs/second. RU is an abstraction of system resources; CPU, memory and IOPS. The cost to read 1 KB item equals 1 RU. In other words, your application will consume 1 RU if it reads a record from CosmosDB with size 1 KB.

Every CRUD (create, read, update, delete) operation on the collection uses a mix of those resources and depending on the operation the consumption will vary. Imagine reading a collection with complex query or reading a single record by ID. Performing a query will consume more RUs then reading a single record by ID within the same collection, because more resources are used. Creating and updating records is also more expensive. Thus this is very important metric for speed, cost and performance of the application you are building on top or Azure CosmosDB.

Before you start you need a CosmosDB account. Read here on how to do that. Then create a database and collection and import some data. Finally open up Azure Portal CosmosDB Data Explorer and open “New SQL Query” editor (yes you can actually write SQL).

Figure 1: Access ComosDb Data Explorer blade in Azure portal.

Read request charge from Data Explorer in Azure Portal. Example from simple select of whole documents on a certain date. — Figure 2: Read request charge from Data Explorer in Azure Portal. Example from simple select of whole documents on a certain date (which is a partition key).

Figure 3: Count documents query with the same condition shows more consumption.

From the screen shots above you can see a difference in RUs consumption. First one only retrieves the records from a single partition (read more here) and consumes roughly 8 RUs, where the second one is doing count through the whole partition and thus a consumption is around 84 RUs. You can tweak, compare and optimize your queries in Data Explorer and once you are satisfied use them in your application.

There is also another way to check RUs, and that’s from code (C#). This has some benefits which I will show you below. See the line 19 of the snippet where I specify variable ru (var ru = documents.RequestCharge;), which is a property (type double) of the FeedResponse object – documents. You could return ru variable as a result, store it to a ViewBag or log, etc.

//Excerpt start
using (client = new DocumentClient(new Uri(_config["CosmosDb:ServerUrl"]), _config["CosmosDb:SecretKey"]))
{
    Database database = await client.CreateDatabaseIfNotExistsAsync(new Database { Id = _config["CosmosDb:Database"] });

    var collectionLink = UriFactory.CreateDocumentCollectionUri(database.Id, _config["CosmosDb:OutDnProcessedDocumentsTable"]);

    IQueryable<ProcessedDocumentViewModel> query;

    var feedOptions = new FeedOptions
    {
        MaxItemCount = documentResourceParameters.PageSize,
        EnableCrossPartitionQuery = true
    };

    //Create document query as IQueryable
    query = client.CreateDocumentQuery<ProcessedDocumentViewModel>(collectionLink, feedOptions).AsQueryable();

    var documents = await query.AsDocumentQuery().ExecuteNextAsync<ProcessedDocumentViewModel>();
    var ru = documents.RequestCharge;
    //return documents.ToList();
    //log...
}
//Excerpt end

//Excerpt start

using (client = new DocumentClient(new Uri(_config["CosmosDb:ServerUrl"]), _config["CosmosDb:SecretKey"]))

{

Database database = await client.CreateDatabaseIfNotExistsAsync(new Database { Id = _config["CosmosDb:Database"] });

var collectionLink = UriFactory.CreateDocumentCollectionUri(database.Id, _config["CosmosDb:OutDnProcessedDocumentsTable"]);

IQueryable<ProcessedDocumentViewModel> query;

var feedOptions = new FeedOptions

{

MaxItemCount = documentResourceParameters.PageSize,

EnableCrossPartitionQuery = true

};

//Create document query as IQueryable

query = client.CreateDocumentQuery<ProcessedDocumentViewModel>(collectionLink, feedOptions).AsQueryable();

var documents = await query.AsDocumentQuery().ExecuteNextAsync<ProcessedDocumentViewModel>();

var ru = documents.RequestCharge;

//return documents.ToList();

//log...

}

//Excerpt end

Another useful feature related to CosmosDB throughput is DisableRUPerMinuteUsage. When you do some sort of a bulk processing (for example during the night) your methods can temporary disable the RU throughput to accommodate the load. You can combine that with the RequestCharge from above to construct an “adaptive algorithm”. Look at the example below where I set DisableRUPerMinuteUsage = true property in FeedOption object (line 12).

public async Task<ProcessedDocumentDto> ReadProcessedDocumentAsync(string documentNumber, string companyId, string collectionName)
{
    using (_Client = new DocumentClient(new Uri(_Url), _SecretKey))
    {
        Database database = await _Client.CreateDatabaseIfNotExistsAsync(new Database { Id = _Database });

        var collectionLink = UriFactory.CreateDocumentCollectionUri(database.Id, collectionName);

        var feedOptions = new FeedOptions
        {
            EnableCrossPartitionQuery = true,
            DisableRUPerMinuteUsage = true
        };

        return _Client.CreateDocumentQuery<ProcessedDocumentDto>(collectionLink, feedOptions)
            .Where(d => d.DocumentNumber == documentNumber)
            .Where(d => d.CompanyName == companyId)
            .FirstOrDefault();
    }
}

public async Task<ProcessedDocumentDto> ReadProcessedDocumentAsync(string documentNumber, string companyId, string collectionName)

{

using (_Client = new DocumentClient(new Uri(_Url), _SecretKey))

{

Database database = await _Client.CreateDatabaseIfNotExistsAsync(new Database { Id = _Database });

var collectionLink = UriFactory.CreateDocumentCollectionUri(database.Id, collectionName);

var feedOptions = new FeedOptions

{

EnableCrossPartitionQuery = true,

DisableRUPerMinuteUsage = true

};

return _Client.CreateDocumentQuery<ProcessedDocumentDto>(collectionLink, feedOptions)

.Where(d => d.DocumentNumber == documentNumber)

.Where(d => d.CompanyName == companyId)

.FirstOrDefault();

}

Thank you for reading and leave a comment below. I hope you like this little nugget :). Till next time.

Check Azure CosmosDB “currency” from code and adapt the throughput

About Miha

Leave a Reply Cancel Reply

Check Azure CosmosDB “currency” from code and adapt the throughput

About Miha

Related Posts

Leave a Reply Cancel Reply