#938

RFC: Add blob feature

Openeibachwants to merge intofrom

This adds the ability to trace the status of arbitrary blobs in memory.

This is an RFC only, to see if I am thinking into the right direction. The encoding parameter is meant to add some decoding ability in the profiler later on. Documentation and timeline integration is still missing completely.

Activity

30
D
d8ea926
wolfpld1y ago

What is the intended use case?

Have you tried sending "large" blobs, i.e. larger than enum { TargetFrameSize = 256 * 1024 } bytes?

eibach1y ago

My use case is recording structs or packets on the fly so I can inspect them later. Intended size is << 64k, with 64k as upper limit.

wolfpld1y ago

I'm not sure about imposing a limit. You may want instead to send the 3D scene geometry for debug purposes, and that could be easily in the megabytes range. (I actually did such a thing: https://bitbucket.org/wolfpld/tracy/commits/branch/mesh-debug)

eibach1y ago

Ok, I see. I have used the string infrastructure because I thought that identical blobs should only get stored and transferred once. Do you think that would also make sense for bigger blobs like the geometry from your usecase?

wolfpld1y ago

Sorry, I don't remember the exact details of how things are transferred, so I can't comment on the specifics.

Hashing things for storage shouldn't be a problem, you can get tens of GB/s of hashing perf e.g. with xxh3.

Hashing things for transfer looks like a no-go for me, as you'd need to do it on the client, and doing extra calculations when you are profiling and want to keep impact minimal is something you'd generally want to avoid.

eibach1y ago

Ah yes, you are right, hashing only is done on the server side. I got that mixed up. I will check if I hit any limits when I crank up blob size.

eibach1y ago

Works only up to 64k - 1 bytes using the string infrastructure. So maybe I have to come up with a different transfer strategy...

eibachreviewed1y ago
eibachreviewed1y ago
eibachreviewed1y ago
eibachreviewed1y ago
wolfpldreviewed1y ago
eibach1y ago

I have added support for fragmented transfer of blob data. Now uint32_t should be the limit ;)

wolfpld1y ago

Why is m_serverQuerySpaceLeft increased? The client should send all the blob fragments in succession, and the server doesn't make any queries?