Long-running operation - October test

A "long-running" test refers to a type of test that is designed to evaluate the system's behavior under sustained use over a long period of time. These tests can be used to identify problems that might not be apparent in short-term testing scenarios.

Among other things, long-running tests are often used to identify memory leaks, resource leaks, or degradation in system performance over time. If a system runs perfectly for an hour but starts having issues after several hours or days, a long-running test would help identify such problems.

In particular, we want to measure CPU and memory usages of Dolos compared to the Haskell node. The main goal of Dolos is to provide a lightweight alternative (with substantially reduced features) for data access use-cases. To perform the comparison, both implementations were hosted under the equivalent conditions:

  • same hardware
  • both fully-synced to the Preview network
  • similar client request profile (1 chain-sync consumer)

CPU Usage

In this analysis we compare CPU usage. It is expressed as shares of a vCPU (~core). 1 share represents 1/1000 of a vCPU. Each bucket represent the average of shares utilized by each process in a 15-minute period. The information was gathered after continuous operations throughout a 24 hr period.

TimeDolosHaskell
2023-10-19 14:45:0026.9302
2023-10-19 15:00:0021.8287
2023-10-19 15:15:0027.8309
2023-10-19 15:30:0028.2303
2023-10-19 15:45:0029.0301
2023-10-19 16:00:0029.3221
2023-10-19 16:15:0030.5327
2023-10-19 16:30:0026.5302
2023-10-19 16:45:0026.9300
2023-10-19 17:00:0028.5307
2023-10-19 17:15:0026.1300
2023-10-19 17:30:0030.8311
2023-10-19 17:45:0022.9285
2023-10-19 18:00:0024.0290
2023-10-19 18:15:0026.5303
2023-10-19 18:30:0022.7287
2023-10-19 18:45:0022.7293
2023-10-19 19:00:0025.7297
2023-10-19 19:15:0021.0291
2023-10-19 19:30:0023.0301
2023-10-19 19:45:0022.5297
2023-10-19 20:00:0023.4307
2023-10-19 20:15:0025.7302
2023-10-19 20:30:0025.0306
2023-10-19 20:45:0025.6302
2023-10-19 21:00:0025.4297
2023-10-19 21:15:0023.5300
2023-10-19 21:30:0025.4301
2023-10-19 21:45:0022.2303
2023-10-19 22:00:0023.9298
2023-10-19 22:15:0026.7296
2023-10-19 22:30:0022.7290
2023-10-19 22:45:0025.5300
2023-10-19 23:00:0022.6300
2023-10-19 23:15:0022.8290
2023-10-19 23:30:0025.4294
2023-10-19 23:45:0023.6301
2023-10-20 00:00:0024.0295
2023-10-20 00:15:0021.9301
2023-10-20 00:30:0022.7290
2023-10-20 00:45:0023.9302
2023-10-20 01:00:0023.7303
2023-10-20 01:15:0023.4288
2023-10-20 01:30:0023.7297
2023-10-20 01:45:0023.5296
2023-10-20 02:00:0027.2303
2023-10-20 02:15:0022.2290
2023-10-20 02:30:0023.9291
2023-10-20 02:45:0023.2289
2023-10-20 03:00:0025.8307
2023-10-20 03:15:0026.3290
2023-10-20 03:30:0025.3300
2023-10-20 03:45:0022.0290
2023-10-20 04:00:0026.6299
2023-10-20 04:15:0026.4301
2023-10-20 04:30:0027.0293
2023-10-20 04:45:0026.8302
2023-10-20 05:00:0031.1309
2023-10-20 05:15:0024.7290
2023-10-20 05:30:0028.1304
2023-10-20 05:45:0024.1282
2023-10-20 06:00:0029.3308
2023-10-20 06:15:0025.3298
2023-10-20 06:30:0024.3287
2023-10-20 06:45:0032.2317
2023-10-20 07:00:0026.8288
2023-10-20 07:15:0027.0295
2023-10-20 07:30:0030.3306
2023-10-20 07:45:0028.6300
2023-10-20 08:00:0032.2316
2023-10-20 08:15:0032.4301
2023-10-20 08:30:0031.0302
2023-10-20 08:45:0041.5310
2023-10-20 09:00:0022.9285
2023-10-20 09:15:0031.8302
2023-10-20 09:30:0028.5285
2023-10-20 09:45:0031.0299
2023-10-20 10:00:0026.1285
2023-10-20 10:15:0032.2309
2023-10-20 10:30:0026.8296
2023-10-20 10:45:0025.3285
2023-10-20 11:00:0033.7306
2023-10-20 11:15:0031.0303
2023-10-20 11:30:0030.9293
2023-10-20 11:45:0030.1308
2023-10-20 12:00:0032.8299
2023-10-20 12:15:0023.9293
2023-10-20 12:30:0029.3302
2023-10-20 12:45:0032.7299
2023-10-20 13:00:0028.7304
2023-10-20 13:15:0032.9307
2023-10-20 13:30:0037.5310
2023-10-20 13:45:0032.0309
2023-10-20 14:00:0032.2298
2023-10-20 14:15:0033.1309
2023-10-20 14:30:0034.2313
2023-10-20 14:45:0035.7308

Memory Usage

In this analysis we compare memory usage. It is expressed as total amount of data (KB, MB, GB). Each bucket represent the average of memory utilized by each process in a 15-minute period. The information was gathered after continuous operations throughout a 24 hr period.

TimeDolosHaskell
2023-10-19 14:45:0062.3 MB2.44 GB
2023-10-19 15:00:0063.2 MB2.44 GB
2023-10-19 15:15:0061.6 MB2.44 GB
2023-10-19 15:30:0062.5 MB2.44 GB
2023-10-19 15:45:0063.3 MB2.44 GB
2023-10-19 16:00:0063.0 MB2.44 GB
2023-10-19 16:15:0063.9 MB2.44 GB
2023-10-19 16:30:0064.4 MB2.44 GB
2023-10-19 16:45:0064.8 MB2.44 GB
2023-10-19 17:00:0064.9 MB2.44 GB
2023-10-19 17:15:0065.2 MB2.44 GB
2023-10-19 17:30:0065.8 MB2.44 GB
2023-10-19 17:45:0066.0 MB2.44 GB
2023-10-19 18:00:0066.1 MB2.44 GB
2023-10-19 18:15:0054.7 MB2.44 GB
2023-10-19 18:30:0055.4 MB2.44 GB
2023-10-19 18:45:0055.9 MB2.44 GB
2023-10-19 19:00:0057.0 MB2.44 GB
2023-10-19 19:15:0057.3 MB2.44 GB
2023-10-19 19:30:0057.6 MB2.44 GB
2023-10-19 19:45:0057.9 MB2.44 GB
2023-10-19 20:00:0058.4 MB2.44 GB
2023-10-19 20:15:0058.8 MB2.44 GB
2023-10-19 20:30:0059.0 MB2.44 GB
2023-10-19 20:45:0059.4 MB2.44 GB
2023-10-19 21:00:0059.6 MB2.44 GB
2023-10-19 21:15:0060.0 MB2.44 GB
2023-10-19 21:30:0060.2 MB2.44 GB
2023-10-19 21:45:0060.5 MB2.44 GB
2023-10-19 22:00:0061.0 MB2.44 GB
2023-10-19 22:15:0061.1 MB2.44 GB
2023-10-19 22:30:0061.5 MB2.44 GB
2023-10-19 22:45:0062.0 MB2.44 GB
2023-10-19 23:00:0062.3 MB2.44 GB
2023-10-19 23:15:0062.6 MB2.44 GB
2023-10-19 23:30:0062.8 MB2.44 GB
2023-10-19 23:45:0063.2 MB2.44 GB
2023-10-20 00:00:0063.4 MB2.44 GB
2023-10-20 00:15:0063.8 MB2.44 GB
2023-10-20 00:30:0064.3 MB2.44 GB
2023-10-20 00:45:0064.6 MB2.44 GB
2023-10-20 01:00:0064.9 MB2.44 GB
2023-10-20 01:15:0065.2 MB2.44 GB
2023-10-20 01:30:0065.4 MB2.44 GB
2023-10-20 01:45:0066.0 MB2.44 GB
2023-10-20 02:00:0066.4 MB2.44 GB
2023-10-20 02:15:0066.7 MB2.44 GB
2023-10-20 02:30:0066.9 MB2.44 GB
2023-10-20 02:45:0067.3 MB2.44 GB
2023-10-20 03:00:0068.0 MB2.44 GB
2023-10-20 03:15:0068.5 MB2.44 GB
2023-10-20 03:30:0069.0 MB2.44 GB
2023-10-20 03:45:0069.2 MB2.44 GB
2023-10-20 04:00:0069.4 MB2.44 GB
2023-10-20 04:15:0069.6 MB2.44 GB
2023-10-20 04:30:0070.1 MB2.44 GB
2023-10-20 04:45:0070.5 MB2.44 GB
2023-10-20 05:00:0070.8 MB2.44 GB
2023-10-20 05:15:0071.2 MB2.44 GB
2023-10-20 05:30:0071.5 MB2.44 GB
2023-10-20 05:45:0071.9 MB2.44 GB
2023-10-20 06:00:0072.2 MB2.44 GB
2023-10-20 06:15:0072.6 MB2.44 GB
2023-10-20 06:30:0072.7 MB2.44 GB
2023-10-20 06:45:0073.0 MB2.44 GB
2023-10-20 07:00:0073.3 MB2.44 GB
2023-10-20 07:15:0073.6 MB2.44 GB
2023-10-20 07:30:0074.0 MB2.44 GB
2023-10-20 07:45:0074.4 MB2.44 GB
2023-10-20 08:00:0074.9 MB2.44 GB
2023-10-20 08:15:0075.4 MB2.44 GB
2023-10-20 08:30:0075.9 MB2.44 GB
2023-10-20 08:45:0079.5 MB2.44 GB
2023-10-20 09:00:0079.6 MB2.44 GB
2023-10-20 09:15:0079.9 MB2.44 GB
2023-10-20 09:30:0080.3 MB2.44 GB
2023-10-20 09:45:0080.9 MB2.44 GB
2023-10-20 10:00:0081.3 MB2.44 GB
2023-10-20 10:15:0082.0 MB2.44 GB
2023-10-20 10:30:0082.4 MB2.44 GB
2023-10-20 10:45:0082.8 MB2.44 GB
2023-10-20 11:00:0083.3 MB2.44 GB
2023-10-20 11:15:0083.4 MB2.44 GB
2023-10-20 11:30:0083.9 MB2.44 GB
2023-10-20 11:45:0084.2 MB2.44 GB
2023-10-20 12:00:0084.7 MB2.44 GB
2023-10-20 12:15:0084.8 MB2.44 GB
2023-10-20 12:30:0085.5 MB2.44 GB
2023-10-20 12:45:0085.8 MB2.44 GB
2023-10-20 13:00:0086.1 MB2.44 GB
2023-10-20 13:15:0086.9 MB2.44 GB
2023-10-20 13:30:0088.1 MB2.44 GB
2023-10-20 13:45:0088.4 MB2.44 GB
2023-10-20 14:00:0088.6 MB2.44 GB
2023-10-20 14:15:0089.0 MB2.44 GB
2023-10-20 14:30:0089.2 MB2.44 GB
2023-10-20 14:45:0089.4 MB2.44 GB