This page contains internal documentation for development.
This repository follows the codesiging.guide in combination with fastlane match. Therefore the sample apps use manual code signing, see fastlane docs:
In most cases, fastlane will work out of the box with Xcode 9 and up if you selected manual code signing and choose a provisioning profile name for each of your targets.
Reach out to @philipphofmann if you need access to the match git repository.
CI runs the unit tests for one job with thread sanitizer enabled to detect race conditions.
The Test scheme of Sentry uses TSAN_OPTIONS
to specify the suppression file to ignore false positives or known issues.
It's worth noting that you can use the $(PROJECT_DIR)
to specify the path to the suppression file.
To run the unit tests with the thread sanitizer enabled in Xcode click on edit scheme, go to tests, then open diagnostics, and enable Thread Sanitizer.
- ThreadSanitizerSuppressions
- Running Tests with Clang's AddressSanitizer
- Diagnosing Memory, Thread, and Crash Issues Early
- Stackoverflow: ThreadSanitizer suppression file with Xcode
The SentryTestLogConfig
sets the log level to debug in load
, so we understand what's going on during out tests.
The clearTestState
method does the same, in case a test changes the log level.
CI runs UI tests on simulators via the test.yml
workflow, and on devices via saucelabs-UI-tests.yml
. All are run for each PR, and Sauce Labs tests also run on a nightly cron schedule.
You can find the available devices on their website. Another way to check their available devices is to go to live app testing, go to iOS-Swift and click on choose device. This brings the full list of devices with more details.
Once daily and for every PR via Github action, the benchmark runs in Sauce Labs, on a high-end device we categorize. Benchmarks run from an XCUITest (PerformanceBenchmarks
target) using the iOS-Swift sample app, under the iOS-Swift-Benchmarking
scheme. PerformanceViewController
provides a start and stop button for controlling when the benchmarking runs, and a text field to marshal observations from within the test harness app into the test runner app. There, we assert that the P90 of all trials remains under 5%. We also print the raw results to the test runner's console logs for postprocessing into reports with //scripts/process-benchmark-raw-results.py
.
- Tap the button to start a Sentry transaction with the associated profiling.
- Run a loop performing large amount of calculations to use as much CPU as possible. This simulates something an app developer would want to profile in a real world scenario.
- While benchmarking, run a sampling profiler at 10 Hz to calculate the CPU usage of each thread, in particular the Sentry profiler's, to calculate its relative usage.
- Tap the button to stop the transaction after waiting for 15 seconds.
- Calculate the total time used by app threads and separately, the profiler's thread. Keep separated by system call and user call times.
- Write these four values as CSV into the text field accessible as an XCUIElement in the runner app.
- Run the procedure 20 times, then assert that the 90th percentile remains under 5% so we can be alerted via CI if it spikes.
- Sauce Labs allows relaxing the timeout for a suite of tests and for a
XCTestCase
subclass' collection of test case methods, but each test case in the suite must run in less than 15 minutes. 20 trials takes too long, so we split it up into multiple test cases, each running a subset of the trials. - This is done by dynamically generating test case methods in
SentrySDKPerformanceBenchmarkTests
, which is necessarily written in Objective-C since this is not possible to do in Swift tests. By doing this dynamically, we can easily fine tune how we split up the work to account for changes in the test duration or in constraints on how things run in Sauce Labs etc.
- Sauce Labs allows relaxing the timeout for a suite of tests and for a
The following script applies a patch so Xcode uploads the iOS-Swift's dSYMs to Sentry during Xcode's build phase. Ensure to not commit the patch file after running this script, which then contains your auth token.
./scripts/upload-dsyms-with-xcode-build-phase.sh YOUR_AUTH_TOKEN
You can use the generate-classes.sh
to generate ViewControllers and other classes to emulate a large project. This is useful, for example, to test the performance of swizzling in a large project without having to check in thousands of lines of code.
The diagrams are created with PlantUml. The advantage of PlantUml over other diagram tools is that you describe the diagrams with text, and you don't have to worry about UML and layout, which can be time-consuming for changes. Furthermore, the diagrams can be stored in git.
With Visual Studio Code and the PlantUml Plugin you can create diagrams, view them and export them. If you don't want to use Visual Studio Code, there are many alternatives.
For learning the syntax and quickly playing around you can check out Plant Text.
Visual Studio Code needs a rendering engine for PlantUml. We recommend using the following Docker image:
docker run -d -p 8080:8080 plantuml/plantuml-server:jetty
To ensure the rendering server is running properly, visit with localhost:8080
.
Finally, you have to configure the rendering server in Visual Studio Code. For this, open the settings of Visual Studio Code. Choose Extensions > PlantUML configuration
. Click on Edit in settings.json
. Then paste the following into the config:
{
"plantuml.server": "http://localhost:8080",
"plantuml.render": "PlantUMLServer"
}
Save the settings and you should be able to render a diagram.
You can find the official guide here: configure a rendering server.