#463

fix(rdb): lowercase filter values for every operator on text fields

OpenRaphaelFakhriwants to merge intofrom

Description of the Change

Problem

TableQuery.Init maps text fields to LOWER(column::text), but DataFilter lowercases a string value only for like and not like. A filter such as {"field":"title","operator":"=","value":"Recon Example"} compares LOWER(title::text) = 'Recon Example' and never matches, and the same value with != matches every row. The in operator already lowercases its items, so the same value behaves differently depending on the operator.

Solution

DataFilter lowercases a string value for every operator when the field is a text field, using the same isNumbericField check that Init uses to decide whether to wrap the column in LOWER(...). Numeric, *_id and *_at fields keep their values unchanged, and like / not like still wrap the value in %.

Closes #462

Type of Change

  • 🐛 Bug fix (non-breaking change which fixes an issue)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔧 Configuration change
  • 🧪 Test update
  • 🛡️ Security update

Areas Affected

  • Core Services (Frontend UI/Backend API)
  • AI Agents (Researcher/Developer/Executor)
  • Security Tools Integration
  • Memory System (Vector Store/Knowledge Base)
  • Monitoring Stack (Grafana/OpenTelemetry)
  • Analytics Platform (Langfuse)
  • External Integrations (LLM/Search APIs)
  • Documentation
  • Infrastructure/DevOps

Testing and Verification

Test Configuration

PentAGI Version: main (843dfeb)
Docker Version: n/a (postgres:16-alpine for the test tier)
Host OS: Linux x86_64
LLM Provider: n/a
Enabled Features: none

Test Steps

  1. Start a PostgreSQL server and set PENTAGI_TEST_DSN.
  2. Run go test -tags postgres -race -count=1 -run TestTable_ ./pkg/server/rdb/ from backend.
  3. Revert table.go and run the same command.

Test Results

The new TestTable_Query_ComparesTextFieldsWhateverTheCaseOfTheValue lives in table_live_test.go (the postgres tier), because Init emits LOWER(column::text) and sqlite cannot parse the ::text cast. It filters a table holding Recon Example and Other through TableQuery.Query with =, !=, like and in.

Without the fix:

--- FAIL: TestTable_Query_ComparesTextFieldsWhateverTheCaseOfTheValue/equal_to_the_stored_title
    table_live_test.go:88: title = Recon Example counted 0 and read [], want [{1 Recon Example}]
--- FAIL: TestTable_Query_ComparesTextFieldsWhateverTheCaseOfTheValue/not_equal_to_the_stored_title
    table_live_test.go:88: title != Recon Example counted 2 and read [{2 Other} {1 Recon Example}], want [{2 Other}]

With the fix, all 4 subtests pass. go test -race ./pkg/server/rdb/ ./pkg/server/services/ passes, and golangci-lint v2.12.2 reports 0 issues on pkg/server/rdb with and without the postgres tag.

Security Considerations

None. The value is still passed as a bound parameter.

Performance Impact

None. One strings.ToLower per text filter value.

Documentation Updates

  • README.md updates
  • API documentation updates
  • Configuration documentation updates
  • GraphQL schema updates
  • Other:

Deployment Notes

None.

Checklist

Code Quality

  • My code follows the project's coding standards
  • I have added/updated necessary documentation
  • I have added tests to cover my changes
  • All new and existing tests pass
  • I have run go fmt and go vet (for Go code)
  • I have run pnpm run lint (for TypeScript/JavaScript code)

Security

  • I have considered security implications
  • Changes maintain or improve the security model
  • Sensitive information has been properly handled

Compatibility

  • Changes are backward compatible
  • Breaking changes are clearly marked and documented
  • Dependencies are properly updated

Documentation

  • Documentation is clear and complete
  • Comments are added for non-obvious code
  • API changes are documented

Additional Notes

Filters that already worked are unchanged: a lower-case value gives the same result as before, and like, not like and in were already lowercased.