Basics Behind the Calculation

Before you learn how to estimate throughput and storage for the service, let's review the throughput and storage unit definitions.

• Write Unit (WU): One Write Unit is defined as throughput for up to 1 kilobyte (KB) of data per second. A write operation is any Oracle NoSQL Database Cloud Service API call that results in insertion, update, or deletion of a record. A NoSQL table has a write limit value which specifies the number of write units that may be used each second. Index updates also consume write units.

  For example, a record size of less than 1 KB requires one WU for a write operation. A record size of 1.5 KB requires two WUs for the write operation.

• Read Unit (RU): One Read Unit is defined as throughput for up to 1 KB of data per second for an eventually consistent read operation. Your NoSQL table has a read limit value which specifies the number of read units that may be used each second.

  For example, a record size of less than 1 KB requires one RU for an eventually consistent read operation. A record size of 1.5 KB requires two RUs for an eventually consistent read operation and four RUs for an absolutely consistent read operation.

• Storage Capacity: One storage unit is a single gigabyte (GB) of data storage.

• Absolute Consistency: The data returned is expected to be the most recently written data to the database.

• Eventual Consistency: The data returned may not be the most recently written data to the database; if no new updates are made to the data, eventually all accesses to that data return the latest updated value.

Factors that Impact the Capacity Unit

Before you provision the capacity units, it is important to consider the following factors that impact the read, write, and storage capacities.

• Record size: As the record size increases, the number of capacity units consumed to write or read data also increases.

• Data consistency: Absolute consistency reads are twice the cost of eventual consistency reads.

• Secondary Indexes: In a table, when an existing record is modified (added, updated, or deleted), updating secondary indexes consume Write Units. The total provisioned throughput cost for a write operation is the sum of write units consumed by writing to the table and updating the local secondary indexes.

• Data modeling choice: With schema-less JSON, each document is self-describing which adds metadata overhead to the overall size of the record. With fixed schema tables, the overhead for each record is exactly 1 byte.

• Query pattern: The cost of a query operation depends on the number of rows retrieved, number of predicates, the size of the source data, projections, and the presence of indexes. The least expensive queries specify a shard key or index key (with an associated index) to allow the system to take advantage of primary and secondary indexes. An application can try different queries and examine the consumed throughput to help tune the operations.

Real World Example: How to Estimate your Application Workload

Consider an E-commerce application example to learn how to estimate reads and writes per second. In this example, Oracle NoSQL Database Cloud Service is used to store the product catalog information of the application.

1. Identify the data model (JSON or fixed-table), record size, and key size for the application.

   Assume that the E-commerce application follows the JSON data model and the developer has created a simple table with two columns. A record identifier (primary key) and a JSON document for the product features and attributes. The JSON document, which is under 1 KB (0.8 KB) is as follows:

   {
     "additionalFeatures": "Front Facing 1.3MP Camera",
     "os": "Macintosh OS X 10.7", 
     "battery": {      
       "type": "Lithium Ion (Li-Ion) (7000 mAH)",
       "standbytime" : "24 hours" },
     "camera": {    
       "features": ["Flash","Video"],
       "primary": "5.0 megapixels" },
     "connectivity": {
       "bluetooth": "Bluetooth 2.1",
       "cell": "T-mobile HSPA+ @ 2100/1900/AWS/850 MHz",
       "gps": true,
       "infrared": false,
       "wifi": "802.11 b/g" },
     "description": "Apple iBook is the best in class computer
       		for your professional and personal work.",
     "display": {
       "screenResolution": "WVGA (1280 x 968)",
       "screenSize": "13.0 inches" },
     "hardware": {
       "accelerometer": true,
       "audioJack": "3.5mm",
       "cpu": "Intel i7 2.5 GHz",
       "fmRadio": false,
       "physicalKeyboard": false,
       "usb": "USB 3.0" },
     "id": "appleproduct_1",
     "images": ["img/apple-laptop.jpg"],
     "name": "Myshop.com : Apple iBook",
     "sizeAndWeight": {
       "dimensions": [
         "300 mm (w)",
         "300 mm (h)",
         "12.4 mm (d)" ],
       "weight": "1250.0 grams" },
     "storage": {
       "hdd": "750GB",
       "ram": "8GB" }
   }

   Assume that the application has 100,000 such records and the primary key is about 20 bytes in size. Also, assume that there are queries that would read records using secondary index. For example, to find all the records that have screen size of 13 inches. So, an index is created on the screenSize field.

   The information in summarized as follows:

   Tables	Rows per Table	Columns per Table	Key Size in Bytes	Value Size in Bytes (sum of all columns)	Indexes	Index Key Size in Bytes
   1	100000	2	20	1 KB	1	20

2. Identify the list of operations (typically CRUD operations and Index reads) on the table and at what rate (per second) they are expected.

   Operation	Number of Operations (per second)	Example
   Create Records.	3	To create a product.
   Read records using the primary key.	200	To read product details using the product ID.
   Read records using the secondary index.	1	To fetch all products that have a screen size of 13 inches.
   Update or add an attribute to a record.	5	To update the product description of a camera or To add information about the weight of a camera.
   Delete record.	5	To delete an existing product.

3. Identify the read and write consumption in KB.

   Operation	Assumptions (If any)	Formula	Read Consumption (KB)	Write Consumption (KB)	Notes/Explanation
   Create Records.	Assume that the records are created without performing any condition checks (if they exist).	Record size (rounded to next KB) + 1 KB(index) * (number of indexes)	0	1 KB + 1 KB (1 ) = 2 KB	Record size is 1 KB ( 0.8 KB for JSON column and 20 bytes for key column) and there is one index of size 1 KB. A create operation incurs a read unit cost if you execute the put commands with some options. Since you need to guarantee that you are reading the most current version of the row, absolute consistent reads are used. In such cases you use the multiplier 2 in the read unit formula. Here are the different options of determining read unit costs: If Option.IfAbsent or Option.IfPresent is used, then Read Consumption = 2 If setReturnRow is used, then Read Consumption = 2 * Record size If Option.IfAbsent and setReturnRow is used, then Read Consumption = 2 * Record size
   Read records using the primary key.		Record size round up to KB	Record size = 1 KB	0	Record size is 1 KB
   Read records using the secondary index.	Assume that 100 records are returned.	record_size * number_of_records_matched	11KB *100 = 100KB 100 KB + 10 KB = 110 KB	0	There is no charge for secondary index. The record size is 1 KB. For 100 records it is 100 KB. Additional 10 KB accounts for variable overhead that may occur depending on the number of batches returned and the size limit set for the query. The overhead is the cost of reading last key in a batch. This is a variable that depends on the maxReadKB and record size. The overhead is up to (numBatches - 1) * key read cost(1KB).
   Update existing records	Assume that the updated record size is the same as the old record size (1 KB).	Read consumption = record_size * 2 Write consumption = original_record_size + new_record_size + 1 KB (index) * (number of writes)	1 KB * 2	1 KB + 1 KB + 1KB(1) *(2) = 4 KB	When rows are updated using a query(SQL statement), then both read and write units are consumed. Depending on the update it may need to read the primary key, secondary key, or even the record itself. Absolute consistent reads are needed to guarantee we are reading the most recent record. Absolute consistency reads are twice the cost of eventual consistency reads. That is the reason for multiplying by 2 in the formula. Read Consumption: No charge for index and Record size is 1 KB. If executing using the option setReturnRow, then Read Consumption = 2 * Record size Write Consumption: The original and new record size is 1 KB and 1KB for one index.
   Delete record		Read consumption = 1 KB (index) * 2 Write consumption = record_size + 1KB (index) * (number_of_indexes)	1 KB (1) *2 = 2 KB	1 KB + 1 KB(1) * (1) = 2 KB	A delete incurs both read and write unit costs. Since you have to guarantee you are looking at the most current version of the row, absolute consistent reads are used, that is the reason to use the 2 multiplier in the read unit formula. If executing using the option setReturnRow, Read Consumption = 2 * Record size. Otherwise, Read Consumption= 1KB for one index Write Consumption: Record size is 1 KB and 1KB for index. No of index is 1.

   Using steps 2 and 3, determine read and write units for the application workload.

   Operations	Rate of Operations	Reads per Second	Writes per Second
   Create records	3	0	6
   Read records using the primary Key	300	300	0
   Read records using the secondary index	10	1100	0
   Update existing record	5	10	20
   Delete record	1	2	2

   Total Read Units: 1412

   Total Write Units: 28

   Therefore, the E-commerce application is estimated to have a workload of 1412 reads per second and 28 writes per second. Download the Capacity Estimator tool available on Oracle Technology Network to input these values and estimate throughput and storage of your application.