#273

Optimize cyclic payloads and add host process monitor

OpenTone730wants to merge intofrom

Summary

  • Replace the cyclic payload endpoint's repeated immutable-string concatenation (quadratic copying) with a repeating-string construction that preserves exact legacy A-Z output in linear time.
  • Add a read-only GET /api/processes/host-monitor snapshot that enumerates device processes and reports non-definitive indicators: inaccessible or missing executable paths, process/executable-name mismatches, and executables under a writable temp directory. It never terminates or modifies processes or makes network/reputation calls; indicators are not malware verdicts.
  • Document the API and optimization; add focused regression tests.

Ten performance opportunities

Ranked for this repository by potential payoff versus effort; estimates are qualitative and should be benchmarked on representative workloads.

  1. Implemented — make cyclic payload generation linear: hexstrike_server.py (/api/payloads/generate, previously a per-character string-concatenation loop). Repeated immutable-string concatenation copies the growing prefix each iteration; string repetition and slicing preserve the same output with linear construction.
  2. Remove request-thread CPU sampling waits: PerformanceMonitor.monitor_system_resources near line 4583, ResourceMonitor.get_current_usage near line 5435, TelemetryCollector.get_system_metrics near line 6761, and /api/processes/dashboard near line 9532 all use psutil.cpu_percent(interval=1), synchronously delaying each call by about one second. Use non-blocking sampling with a documented warm-up / cached snapshot.
  3. Avoid probing every installed tool serially on /health: health_check near line 9099 loops through the full tool inventory and invokes execute_command for each tool. Resolve executable availability in a batch or maintain a refreshed capability snapshot.
  4. Bound socket enumeration in the dashboard: /api/processes/dashboard near line 9534 obtains the full host connection table just to compute its length. Use a cheaper count or cache/refresh it off the request path where supported.
  5. Avoid full-cache scans during periodic TTL cleanup: AdvancedCache._cleanup_expired near line 5181 walks every expiry every minute. An expiry heap/timing wheel or incremental cleanup bounds pause cost for large caches.
  6. Avoid linear scans for each LRU eviction: AdvancedCache._evict_lru near line 5168 calls min across all access timestamps. An ordered mapping or linked LRU structure makes eviction constant time.
  7. Make cache eviction metadata thread-safe: HexStrikeCache near line 6672 mutates its OrderedDict and statistics without a lock while command requests can run concurrently. A small lock avoids races/retries that hurt throughput under concurrent load.
  8. Reduce peak memory when generating random payloads: /api/payloads/generate near line 9247 builds a list via random.choices and then joins it. A chunked or generator-based implementation can avoid the intermediate list for large allowed payloads.
  9. Reuse HTTP connections and bound repeated CVE lookups: NVD/GitHub/MSF query sites near lines 5989, 6099, 6172, and 6418–6538 create individual requests.get calls. Shared sessions and cache reuse can cut connection setup while retaining external rate limits.
  10. Batch repeated result-cache eviction: HexStrikeCache.set near line 6712 removes entries one by one in a while loop when at capacity. If capacity becomes configurable or multiple entries need clearing, bulk/pruned eviction avoids repeated bookkeeping.

Validation

  • python -m unittest discover -s tests -p test_payload_and_host_process_monitor.py — 3 tests passed, including exact cyclic payload content at boundary and larger sizes, heuristic classification, and GET-only route behavior.
  • python -m py_compile hexstrike_server.py tests\test_payload_and_host_process_monitor.py — passed.
  • git diff --check — passed.