#81

Aggregation queries on in-memory index namespaces (non-flash `index-type`) can silently drop hot records

OpenShubhamTapleopened 4mo ago

Summary

Aggregation queries on in-memory index namespaces (non-flash index-type) can silently drop hot records from results and may still crash the node on index ref-count overflow.

Flash index namespaces (index-type flash) are unaffected, they already use a digest-only collection path with no ref-count pinning.

Severity

incorrect analytics/query results with no client error; potential node crash via cf_assert on ref-count overflow.

Root cause

During aggregation query collection, in-memory namespaces pin records via as_index_reserve() so index entries stay alive until the UDF aggregation stream processes them.

To avoid uint16_t ref-count overflow (as_index.rc), records with rc > 50000 are skipped entirely — but the caller still counts them as succeeded.

1. Silent record drop — as/src/query/query.c

In aggr_query_add_digest():

else {
    if (r->rc > 50000) {
        return; // hack - avoid ref-count overflow
    }

    as_index_reserve(r);
    keys_arr->u.handles[keys_arr->num] = (uint64_t)r_ref->r_h;
}

keys_arr->num++;

Activity

0

No activity yet.