At Backtrace, we aim to release new features in our software every few weeks. Back in early September, we released the new Query Builder tool, Scheduled e-mail reports, and new workflow integrations with FogBugz and Webhooks. Check out the blog post here.
The recent release of Backtrace adds an impressive number of new capabilities, most of them prioritized based on direct feedback from our most frequent users. These include:
- A Data Scrubber that can be used to remove Personally Identifiable Information (PII) from crash reports.
- Support for file attachments that are submitted via ptrace or the HTTP API to be displayed and downloaded from the web console.
- Improved Symbol Management, including a missing symbol notification, new upload wizard, and binary symbol format for improved performance when loading the web debugger.
- Performance improvements throughout the product, including improved responsiveness of the UI for large data sets and an optional query results compression option to minimize the amount of data sent over the wire.
- A new “Copy Callstack to Clipboard” function in the web debugger.
- User Management now displays the count of registered users, and a separate tab for Technical Support accounts that the Backtrace team uses to support your instance has been exposed.
- More configurable workflow integrations allow you to choose which elements are pushed (Classifiers, Callstack, and specific Attributes)
- General styling improvements in the UI for various elements.
- Note: the recent introductions of Scheduled Reports and the Query Builder tools has led us to retire the now obsolete Stability Report.
Let’s review some of these in more detail:
Major New Features
PII Data Scrubbers
Regulated industries, like healthcare and financial services, are required to anonymize data such as credit card numbers and medical records. In addition, many consumer focused companies want to ensure data is anonymized to adhere to corporate policies, and protect against data leaks.
This need led us to develop Data Scrubbers. Data scrubbers provide administrative users with the ability to remove sensitive data, such as Personally Identifiable Information (PII), from submissions to Backtrace before they are stored in the system.
Backtrace provides built in scrubbers for credit cards, social security numbers, encryption key or environment variables.
New scrubbers can be created using regular expressions to identify patterns of text to remove (i.e. IP Address can be scrubbed with regexp='[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+’)
The system will scrub submitted dump file, including execution path, memory, environment variables, register values and user defined attributes. After scrubbing, PII data will be replaced with — characters and then stored on disk.
Data Scrubbers can be configured with the Backtrace morgue command line tool.
Many of our customers have asked for the ability to associate file attachments with crashes being managed in Backtrace. File attachments allow ancillary information, such as log files of user activity, or configuration files with specified properties, to be stored alongside the crash record.
We are excited to announce the first part of this capability is available for use with the 2.40.0 release of the Backtrace Console / 1.19.12 release of coronerd.
- With this release, users will be able to view attachments that have been submitted using the
--attachcommand-line option when using our
ptracedebugger, or our HTTP API.
- Users can view and download attachments via the Web Console. Look for the paperclip icon to see a list of attachments or to upload an attachment.
- We are working closely with the Crashpad team to allow attachments to be a first-class citizen in the Crashpad API. We intend to provide another update with this capability in the next few weeks.
One of most time-consuming portions of analyzing crash data is ensuring that the system has appropriate symbols. This allows users to have human readable debug information. Without this information, deduplication is less accurate, and debugging becomes more difficult. Instead of seeing an accurate grouping of crashes and useful information like function names, parameters, and even variable information for the offending software, you have gaps.
Backtrace now provides advanced symbolication techniques to support analysis of crash dumps that require symbol files.
- Backtrace will notify users when there are crashes missing symbols, and guide them through a Symbol Upload wizard.
- A new Binary Symbol Format significantly improves the speed for processing symbols and for loading the debugger with human readable call stacks.
- An improved user experience makes it easier to generally manage symbols in Backtrace.
- Backtrace also supports on demand retrieval for 3rd party dependent libraries from Microsoft, Electron, Mozilla and Apple. We can also integrate to your private symbol server, and support your build processes to push symbols to Backtrace when they are generated.
Check out more information about Symbol Management in our documentation here.
There are many other enhancements we have been working on, such as improved memory utilization for large object counts, significant improvements to dump processing throughput (in excess of 300% for many crashes) and better deduplication for CEF, CryEngine, Electron, LUA, macOS and wxWidgets.
Copy Callstack to Clipboard
If you are a frequent user of Backtrace, you will appreciate the new “Copy Callstack to Clipboard” function that is now exposed in the web debugger.
[ 0 ] libc-2.19.so __GI___strcmp_ssse3 ( strcmp.S:210 )
[ 1 ] coronerd bpg_object_iterate ( bpg.c:1180 )
[ 2 ] coronerd handle_action_get ( bpg.c:712 )
[ 3 ] coronerd handle_action ( bpg.c:831 )
[ 4 ] coronerd cr_bpg_handler_bpg ( bpg.c:972 )
[ 5 ] coronerd route_task ( http.c:26 )
[ 6 ] coronerd cr_pu_ptp_drain ( pu.c:792 )
[ 7 ] coronerd cr_io_enter ( io.c:401 )
[ 8 ] coronerd cr_pu_enter ( pu.c:1390 )
[ 9 ] coronerd main ( coronerd.c:388 )
More Workflow Customizations
Be sure to also check out the new customization capabilities for workflow integrations. Many users will appreciate the flexibility to choose which attributes and other data are sent with the message payloads.
The Release Notes will give you a nice overview of all the enhancements and bug fixes made in the system over the last month.
Our hosted customers are already using many of these new capabilities . If you deployed Backtrace on premises, please work with your customer success representative to ensure you upgrade to the most recent versions of the product.