Saving and loading Profiler data

The Unity Profiler currently has a few fairly significant pitfalls when it comes to saving and loading Profiler data:

• Only 300 frames are visible in the Profiler window at once
• There is no way to save Profiler data through the user interface
• Profiler binary data can be saved into a file the Script code, but there is no built-in way to view this data

These issues make it very tricky to perform large-scale or long-term testing with the Unity Profiler. They have been raised in Unity's Issue Tracker tool for several years, and there doesn't appear to be any salvation in sight. So, we must rely on our own ingenuity to solve this problem.

Fortunately, the Profiler class exposes a few methods that we can use to control how the Profiler logs information:

1. The Profiler.enabled method can be used to enable/disable the Profiler, which is the equivalent of clicking on the Record button in the Control View of the Profiler.

   Note

   Note that changing Profiler.enabled does not change the visible state of the Record button in the Profiler's Controls bar. This will cause some confusing conflicts if we're controlling the Profiler through both code and the user interface at the same time.

2. The Profiler.logFile method sets the current path of the log file that the Profiler prints data out to. Be aware that this file only contains a printout of the application's frame rate over time, and none of the useful data we normally find in the Profiler's Timeline View. To save that kind of data as a binary file, we must use the options that follow.
3. The Profiler.enableBinaryLog method will enable/disable logging of an additional file filled with binary data, which includes all of the important values we want to save from the Timeline and Breakdown Views. The file location and name will be the same as the value of Profiler.logFile, but with .data appended to the end.

With these methods, we can generate a simple data-saving tool that will generate large amounts of Profiler data separated into multiple files. With these files, we will be able to peruse them at a later date.

Saving Profiler data

In order to create a tool that can save our Profiler data, we can make use of a Coroutine. A typical method will be executed from beginning to end in one sitting. However, Coroutines are useful constructs that allow us write methods that can pause execution until a later time, or an event takes place. This is known as yielding, and is accomplished with the yield statement. The type of yield determines when execution will resume, which could be one of the following types (the object that must be passed into the yield statement is also given):

• After a specific amount of time (WaitForSeconds)
• After the next Update (WaitForEndOfFrame)
• After the next Fixed Update (WaitForFixedUpdate)
• Just prior to the next Late Update (null)
• After a WWW object completes its current task, such as downloading a file (WWW)
• After another Coroutine has finished (a reference to another Coroutine)

The Unity Documentation on Coroutines and Execution Order provides more information on how these useful tools function within the Unity Engine:

• http://docs.unity3d.com/Manual/Coroutines.html
• http://docs.unity3d.com/Manual/ExecutionOrder.html

Tip

Coroutines should not be confused with threads, which execute independently of the main Unity thread. Coroutines always run on the main thread with the rest of our code, and simply pause and resume at certain moments, depending on the object passed into the yield statement.

Getting back to the task at hand, the following is the class definition for our ProfilerDataSaverComponent, which makes use of a Coroutine to repeat an action every 300 frames:

using UnityEngine;
using System.Text;
using System.Collections;

public class ProfilerDataSaverComponent : MonoBehaviour {

  int _count = 0;

  void Start() {
    Profiler.logFile = "";
  }

  void Update () {
    if (Input.GetKey (KeyCode.LeftControl) && Input.GetKeyDown (KeyCode.H)) {
      StopAllCoroutines();
      _count = 0;
      StartCoroutine(SaveProfilerData());
    }
  }

  IEnumerator SaveProfilerData() {
    // keep calling this method until Play Mode stops
    while (true) {

      // generate the file path
      string filepath = Application.persistentDataPath + "/profilerLog" + _count;

      // set the log file and enable the profiler
      Profiler.logFile = filepath;
      Profiler.enableBinaryLog = true;
      Profiler.enabled = true;

      // count 300 frames
      for(int i = 0; i < 300; ++i) {

        yield return new WaitForEndOfFrame();

        // workaround to keep the Profiler working
        if (!Profiler.enabled)
          Profiler.enabled = true;
      }

      // start again using the next file name
      _count++;
    }
  }
}

Try attaching this Component to any GameObject in the Scene, and press Ctrl + H (OSX users will want to replace the KeyCode.LeftControl code with something such as KeyCode.LeftCommand). The Profiler will start gathering information (whether or not the Profiler Window is open!) and, using a simple Coroutine, will pump the data out into a series of files under wherever Application.persistantDataPath is pointing to.

Tip

Note that the location of Application.persistantDataPath varies depending on the Operating System. Check the Unity Documentation for more details at http://docs.unity3d.com/ScriptReference/Application-persistentDataPath.html.

It would be unwise to send the files to Application.dataPath, as it would put them within the Project Workspace. The Profiler does not release the most recent log file handle if we stop the Profiler or even when Play Mode is stopped. Consequently, as files are generated and placed into the Project workspace, there would be a conflict in file accessibility between the Unity Editor trying to read and generate complementary metadata files, and the Profiler keeping a file handle to the most recent log file. This would result in some nasty file access errors, which tend to crash the Unity Editor and lose any Scene changes we've made.

When this Component is recording data, there will be a small overhead in hard disk usage and the overhead cost of IEnumerator context switching every 300 frames, which will tend to appear at the start of every file and consume a few milliseconds of CPU (depending on hardware).

Each file pair should contain 300 frames worth of Profiler data, which skirts around the 300 frame limit in the Profiler window. All we need now is a way of presenting the data in the Profiler window.

Here is a screenshot of data files that have been generated by ProfilerDataSaverComponent:

Note

Note that the first file may contain less than 300 frames if some frames were lost during Profiler warm up.

Loading Profiler data

The Profiler.AddFramesFromFile() method will load a given profiler log file pair (the text and binary files) and append it into the Profiler timeline, pushing existing data further back in time. Since each file will contain 300 frames, this is perfect for our needs, and we just need to create a simple EditorWindow class that can provide a list of buttons to load the files into the Profiler.

Tip

Note that AddFramesFromFile() only requires the name of the original log file. It will automatically find the complimentary binary .data file on its own.

The following is the class definition for our ProfilerDataLoaderWindow:

using UnityEngine;
using UnityEditor;
using System.IO;
using System.Collections;
using System.Collections.Generic;
using System.Text.RegularExpressions;

public class ProfilerDataLoaderWindow : EditorWindow {

  static List<string> s_cachedFilePaths;
  static int s_chosenIndex = -1;

  [MenuItem ("Window/ProfilerDataLoader")]
  static void Init() {
    ProfilerDataLoaderWindow window = (ProfilerDataLoaderWindow)EditorWindow.GetWindow (typeof(ProfilerDataLoaderWindow));
    window.Show ();

    ReadProfilerDataFiles ();
  }

  static void ReadProfilerDataFiles() {
    // make sure the profiler releases the file handle
    // to any of the files we're about to load in
    Profiler.logFile = "";

    string[] filePaths = Directory.GetFiles (Application.persistentDataPath, "profilerLog*");

    s_cachedFilePaths = new List<string> ();

    // we want to ignore all of the binary
    // files that end in .data. The Profiler
    // will figure that part out
    Regex test = new Regex (".data$");

    for (int i = 0; i < filePaths.Length; i++) {
      string thisPath = filePaths [i];

      Match match = test.Match (thisPath);

      if (!match.Success) {
        // not a binary file, add it to the list
        Debug.Log ("Found file: " + thisPath);
        s_cachedFilePaths.Add (thisPath);
      }
    }

    s_chosenIndex = -1;
  }

  void OnGUI () {
    if (GUILayout.Button ("Find Files")) {
      ReadProfilerDataFiles();
    }

    if (s_cachedFilePaths == null)
      return;

    EditorGUILayout.Space ();

    EditorGUILayout.LabelField ("Files");

    EditorGUILayout.BeginHorizontal ();

    // create some styles to organize the buttons, and show
    // the most recently-selected button with red text
    GUIStyle defaultStyle = new GUIStyle(GUI.skin.button);
    defaultStyle.fixedWidth = 40f;

    GUIStyle highlightedStyle = new GUIStyle (defaultStyle);
    highlightedStyle.normal.textColor = Color.red;

    for (int i = 0; i < s_cachedFilePaths.Count; ++i) {

      // list 5 items per row
      if (i % 5 == 0) {
        EditorGUILayout.EndHorizontal ();
        EditorGUILayout.BeginHorizontal ();
      }

      GUIStyle thisStyle = null;

      if (s_chosenIndex == i) {
        thisStyle = highlightedStyle;
      } else {
        thisStyle = defaultStyle;
      }

      if (GUILayout.Button("" + i, thisStyle)) {
        Profiler.AddFramesFromFile(s_cachedFilePaths[i]);

        s_chosenIndex = i;
      }
    }

    EditorGUILayout.EndHorizontal ();
  }
}

The first step in creating any custom EditorWindow is creating a menu entry point with a [MenuItem] attribute and then creating an instance of a Window object to control. Both of these occur within the Init() method.

We're also calling the ReadProfilerDataFiles() method during initialization. This method reads all files found within the Application.persistantDataPath folder (the same location our ProfilerDataSaverComponent saves data files to) and adds them to a cache of filenames to use later.

Finally, there is the OnGUI() method. This method does the bulk of the work. It provides a button to reload the files if needed, verifies that the cached filenames have been read, and provides a series of buttons to load each file into the Profiler. It also highlights the most recently clicked button with red text using a custom GUIStyle, making it easy to see which file's contents are visible in the Profiler at the current moment.

The ProfilerDataLoaderWindow can be accessed by navigating to Window | ProfilerDataLoader in the Editor interface, as show in the following screenshot:

Here is a screenshot of the display with multiple files available to be loaded. Clicking on any of the numbered buttons will push the Profiler data contents of that file into the Profiler.

The ProfilerDataSaverComponent and ProfilerDataLoaderWindow do not pretend to be exhaustive or feature-rich. They simply serve as a springboard to get us started if we wish to take the subject further. For most teams and projects, 300 frames worth of short-term data is enough for developers to acquire what they need to begin making code changes to fix the problem.