#1009

[Draft] Resolution of symbols by the server

DraftGabrielSiliceumwants to merge intofrom

The main features in this pull request is to make the server be able to locally resolve the symbol and serialize them in trace file. For this the client sends his image data to the server.

Why this can be a improvement

  • The client has less work to do.
  • You can resolve symbols in the profiler even if the client machine doesn't have the files containing debug information
  • Less bandwidth used for communication between client and server over time (except at the begin of the recording).
  • Working on unification of the symbols resolution code.
  • By disabling automatic symbol resolution entirely, this can speed up the capture time and size.

Implementation details

  • In the global settings you can specify symbol resolution behaviour (Attempt profiler resolution by profiler, Prevent resolution by application ).
  • Tracy_callstack is now used by Tracy server (in the common folder).
  • Tracy_callstack can use debug GUIDs sent by the client to locate and load symbol information.
  • Only working for Windows right now
    • Designed to work for Linux/mac later on
    • On linux, we already store the GNU build id, so this will be forward compatible
  • Client caches its image information and sends it to the server after the handshake. If new images are loaded later on, a symbol query will be issued (as before) leading to discovery of the module and its debug id will be sent to the server.

Areas of improvements/TODO list

  • Make the resolution of symbols on the server asynchronous (similarly to the symbol worker of the client).
  • Attempt to load debug information using a user-provided path.
  • Provide a way for the user to specify the symbols location from the UI (we can with this make the server totally independent of the client for image caching).
  • ELF/Macho-o resolution by modifying libbacktrace to support resolution of another process. (Apparently not that many modifications required)
  • Rework the content of Tracy_callstack.cpp to provide an API without global state (ie: some "context"/"state" that we could store in the Worker instead of relying on a global cache)
  • Offline symbol resolution (the update program) should use the same mechanism.
    • For now it only uses the image path, never the GUID. This means that you could in theory end up using incorrect symbols (if program was recompiled, OS kernel modules updated, etc).
    • It uses addr2line which is slow and should only serve as a fallback mechanism, after having confirmed the path to the executable matches the one from the capture

The main idea for this feature is to make Tracy more flexible, especially when you can't or don't want the application make the symbol resolution job.

Activity

62
tackyunicorn1y ago

Is there a plan to support caching of source files from symbols that contain source-control information?

In the latest version of Tracy, I see source files cached from symbols that I've generated locally, but not from those that come from a symbol store. So, I have to rely on source location substitutions for these files, which may not always be an accurate representation of the source control version.

P.S. - Thank you for your amazing work on tracy! I can't wait to try the changes from this PR!

Lectem1y ago

In the latest version of Tracy, I see source files cached from symbols that I've generated locally, but not from those that come from a symbol store. So, I have to rely on source location substitutions for these files, which may not always be an accurate representation of the source control version.

Yes, the plan is to be able to offload as much work as possible for offline resolution (as in, we would still keep current behavior for those who need it, but also allow further offline processing. And for those who want full offline resolution, they could). But I'd like to finish the symbols part before tackling source/binaries.

Once we have the debug symbols information we can then query not only symbols but also source files or even binaries themselves. This would also allow for optimisations : look for files on the server (profiler gui) side locally (and decide whether they are the correct version), then fallback to symbol servers (with their cache).

Lectem1y ago

@wolfpld would you like us to split this PR into smaller chunks? It could me reviewing easier

wolfpld1y ago

It would probably be helpful.

Lectem1y ago

I'm turning this PR into a draft as I will be submitting smaller PRs that split the review work

This pull request is a draft

Merging is unavailable until it is marked ready for review.