Callgrind Profile Format, Version 1

(A synonymous name is "Calltree Profile Format". These names actually mean the same since Callgrind was previously named Calltree)

1. Overview

It is used by Callgrind, and derived from the format used by the Cachegrind tool part of Valgrind: it is kept upwards compatible, i.e. data written by Cachegrind can be loaded into KCachegrind without any problems.

This chapter gives on overview of format features and examples. For detailed syntax, look at the format reference below. The reference also contains information about which version of KCachegrind supports which format feature.

1.1 Basic Structure

To uniquely specify that a file is a callgrind profile, it should add "# callgrind format" as first line. This is optional but recommended for easy format detection.

Each file has a header part of an arbitrary number of lines of the format "key: value". The lines with key "positions" and "events" define the meaning of cost lines in the second part of the file: the value of "positions" is a list of subpositions, and the value of "events" is a list of event type names. Cost lines consist of subpositions followed by 64-bit counters for the events, in the order specified by the "positions" and "events" header line.

The "events" header line is always required in contrast to the optional line for "positions", which defaults to "line", i.e. a line number of some source file. In addition, the second part of the file contains position specifications of the form "spec=name". "spec" can be e.g. "fn" for a function name or "fl" for a file name. Cost lines are always related to the function/file specifications given directly before.

1.2 Simple Example

 # callgrind format
 events: Cycles Instructions Flops
 fl=file.f
 fn=main
 15 90 14 2
 16 20 12

The above example gives profile information for event types "Cycles", "Instructions", and "Flops". Thus, cost lines give the number of CPU cycles passed by, number of executed instructions, and number of floating point operations executed while running code corresponding to some source position. As there is no line specifying the value of "positions", it defaults to "line", which means that the first number of a cost line is always a line number.

Thus, the first cost line specifies that in line 15 of source file "file.f" there is code belonging to function "main". While running, 90 CPU cycles passed by, and 2 of the 14 instructions executed were floating point operations. Similarily, the next line specifies that there were 12 instructions executed in the context of function "main" which can be related to line 16 in file "file.f", taking 20 CPU cycles. If a cost line specifies less event counts than given in the "events" line, the rest is assumed to be zero. I.e., there was no floating point instruction executed relating to line 16.

Note that regular cost lines always give self (also called exclusive) cost of code at a given position. If you specify multiple cost lines for the same position, these will be summed up. On the other hand, in the example above there is no specification of how many times function "main" actually was called: profile data only contains sums.

1.3 Associations

The most important extension to the original format of Cachegrind is the ability to specify call relationship among functions. More generally, you specify assoziations among positions. For this, the second part of the file also can contain assoziation specifications. These look similar to position specifications, but consist of 2 lines. For calls, the format looks like

 calls=(Call Count) (Destination position)
 (Source position) (Inclusive cost of call)

The destination only specifies subpositions like line number. Therefore, to be able to specify a call to another function in another source file, you have to precede the above lines with a "cfn=" specification for the name of the called function, and a "cfl=" specification if the function is in another source file. The 2nd line looks like a regular cost line with the difference that inclusive cost spent inside of the function call has to be specified.

Other assoziations are for example (conditional) jumps. See the reference below for details.

1.4 Extended Example

The following example shows 3 functions, "main", "func1", and "func2". Function "main" calls "func1" once and "func2" 3 times. "func1" calls "func2" 2 times.

 # callgrind format
 events: Instructions

 fl=file1.c
 fn=main
 16 20
 cfn=func1
 calls=1 50
 16 400
 cfl=file2.c
 cfn=func2
 calls=3 20
 16 400

 fn=func1
 51 100
 cfl=file2.c
 cfn=func2
 calls=2 20
 51 300

 fl=file2.c
 fn=func2
 20 700

One can see that in "main" only code from line 16 is executed where also the other functions are called. Inclusive cost of "main" is 820, which is the sum of self cost 20 and costs spent in the calls.

Function "func1" is located in "file1.c", the same as "main". Therefore, a "cfl=" specification for the call to "func1" is not needed. The function "func1" only consists of code at line 51 of "file1.c", where "func2" is called.

1.5 Name Compression

With the introduction of association specifications like calls it is needed to specify the same function or same file name multiple times. As absolute filenames or symbol names in C++ can be quite long, it is advantageous to be able to specify integer IDs for position specifications.

To support name compression, a position specification can be not only of the format "spec=name", but also "spec=(ID) name" to specify a mapping of an integer ID to a name, and "spec=(ID)" to reference a previously defined ID mapping. There is a separate ID mapping for each position specification, i.e. you can use ID 1 for both a file name and a symbol name.

With string compression, the example from 1.4 looks like this:

 # callgrind format
 events: Instructions

 fl=(1) file1.c
 fn=(1) main
 16 20
 cfn=(2) func1
 calls=1 50
 16 400
 cfl=(2) file2.c
 cfn=(3) func2
 calls=3 20
 16 400

 fn=(2)
 51 100
 cfl=(2)
 cfn=(3)
 calls=2 20
 51 300

 fl=(2)
 fn=(3)
 20 700

As position specifications carry no information themself, but only change the meaning of subsequent cost lines or associations, they can appear everywhere in the file without any negative consequence. Especially, you can define name compression mappings directly after the header, and before any cost lines. Thus, the above example can also be written as

 # callgrind format
 events: Instructions

 # define file ID mapping
 fl=(1) file1.c
 fl=(2) file2.c
 # define function ID mapping
 fn=(1) main
 fn=(2) func1
 fn=(3) func2

 fl=(1)
 fn=(1)
 16 20
 cfn=(2)
 calls=1 50
 16 400
 cfl=(2)
 cfn=(3)
 calls=3 20
 16 400

 fn=(2)
 51 100
 cfl=(2)
 cfn=(3)
 calls=2 20
 51 300

 fl=(2)
 fn=(3)
 20 700

1.6 Subposition Compression

If a Callgrind data file should hold costs for each assembler instruction of a program, you specify subpostion "instr" in the "positions:" header line, and each cost line has to include the address of some instruction. Addresses are allowed to have a size of 64bit to support 64bit architectures. This motivates for subposition compression: instead of every cost line starting with a 16 character long address, one is allowed to specify relative subpositions.

A relative subposition always is based on the corresponding subposition of the last cost line, and starts with a "+" to specify a positive difference, a "-" to specify a negative difference, or consists of "*" to specify the same subposition. Assume the following example (subpositions can always be specified as hexadecimal numbers, beginning with "0x"):

 # callgrind format
 positions: instr line
 events: ticks

 fn=func
 0x80001234 90 1
 0x80001237 90 5
 0x80001238 91 6

With subposition compression, this looks like

 # callgrind format
 positions: instr line
 events: ticks

 fn=func
 0x80001234 90 1
 +3 * 5
 +1 +1 6

Remark: For assembler annotation to work, instruction addresses have to be corrected to correspond to addresses found in the original binary. I.e. for relocatable shared objects, often a load offset has to be subtracted.

1.7 Miscellaneous

Cost Summary Information

For the visualization to be able to show cost percentage, a sum of the cost of the full run has to be known. Usually, it is assumed that this is the sum of all cost lines in a file. But sometimes, this is not correct. Thus, you can specify a "summary:" line in the header giving the full cost for the profile run. This has another effect: a import filter can show a progress bar while loading a large data file if he knows to cost sum in advance.

Long Names for Event Types and inherited Types

Event types for cost lines are specified in the "events:" line with an abbreviated name. For visualization, it makes sense to be able to specify some longer, more descriptive name. For an event type "Ir" which means "Instruction Fetches", this can be specified the header line

 event: Ir : Instruction Fetches
 events: Ir Dr

In this example, "Dr" itself has no long name assoziated. The order of "event:" lines and the "events:" line is of no importance. Additionally, inherited event types can be introduced for which no raw data is available, but which are calculated from given types. Suppose the last example, you could add

 event: Sum = Ir + Dr

to specify an additional event type "Sum", which is calculated by adding costs for "Ir and "Dr".

2. Reference

2.1 Grammar

A Callgrind profile data file is an expression of "ProfileDataFile" in the following grammar ("+" means "at least once", "?" optional).

 ProfileDataFile := FormatSpec? FormatVersion? Creator? PartData*

 FormatSpec := "# callgrind format\n"

 FormatVersion := "version:" Space* Number "\n"

 Creator := "creator:" NoNewLineChar* "\n"

 PartData := (HeaderLine "\n")+ (BodyLine "\n")+

 HeaderLine := (empty line)
  | ('#' NoNewLineChar*)
  | PartDetail
  | Description
  | EventSpecification
  | CostLineDef

 PartDetail := TargetCommand | TargetID

 TargetCommand := "cmd:" Space* NoNewLineChar*

 TargetID := ("pid"|"thread"|"part") ":" Space* Number

 Description := "desc:" Space* Name Space* ":" NoNewLineChar*

 EventSpecification := "event:" Space* Name InheritedDef? LongNameDef?

 InheritedDef := "=" InheritedExpr

 InheritedExpr := Name
  | Number Space* ("*" Space*)? Name
  | InheritedExpr Space* "+" Space* InheritedExpr

 LongNameDef := ":" NoNewLineChar*

 CostLineDef := "events:" Space* Name (Space+ Name)*
  | "positions:" "instr"? (Space+ "line")?

 BodyLine := (empty line)
  | ('#' NoNewLineChar*)
  | CostLine
  | PositionSpecification
  | AssoziationSpecification

 CostLine := SubPositionList Costs?

 SubPositionList := (SubPosition+ Space+)+

 SubPosition := Number | "+" Number | "-" Number | "*"

 Costs := (Number Space+)+

 PositionSpecification := Position "=" Space* PositionName

 Position := CostPosition | CalledPosition

 CostPosition := "ob" | "fl" | "fi" | "fe" | "fn"

 CalledPosition := " "cob" | "cfl" | "cfn"

 PositionName := ( "(" Number ")" )? (Space* NoNewLineChar* )?

 AssoziationSpecification := CallSpezification
  | JumpSpecification

 CallSpecification := CallLine "\n" CostLine

 CallLine := "calls=" Space* Number Space+ SubPositionList

 JumpSpecification := ...

 Space := " " | "\t"

 Number := HexNumber | (Digit)+

 Digit := "0" | ... | "9"

 HexNumber := "0x" (Digit | HexChar)+

 HexChar := "a" | ... | "f" | "A" | ... | "F"

 Name = Alpha (Digit | Alpha)*

 Alpha = "a" | ... | "z" | "A" | ... | "Z"

 NoNewLineChar := all characters without "\n"

A profile data file ("ProfileDataFile") starts with basic information such as a format marker, the version and creator information, and then has a list of parts, where each part has its own header and body. Parts typically are different threads and/or time spans/phases within a profiled application run.

Note that callgrind_annotate currently only supports profile data files with one part. Callgrind may produce multiple parts for one profile run, but defaults to one output file for each part.

2.2 Description of Header Lines

Basic information in the first lines of a profile data file:

• # callgrind format [Callgrind]
  This line specifies that the file is a callgrind profile, and it has to be the first line. It was added late to the format (with Valgrind 3.13) and is optional, as all readers also should work with older callgrind profiles not including this line. However, generation of this line is recommended to allow desktop environments and file managers to uniquely detect the format.

• version: number [Callgrind]
  This is used to distinguish future profile data file formats. A version of 1 is supposed to be upwards compatible with the Cachegrind format. Optional. If not given, version 1 is assumed. Otherwise, it has to follow directly after the format specification (i.e. be the first line if the optional format specification is skipped)

• creator: string [Callgrind]
  Information about the creator of this file, i.e. the profile tool or conversion script. Optional.

• pid: process id [Callgrind]
  This specifies the process ID of the supervised application for which this profile was generated. Optional.

• cmd: program name + args [Cachegrind]
  This specifies the full command line of the supervised application for which this profile was generated. Optional.

• part: number [Callgrind]
  This specifies a sequentially running number for profile dumps belonging to the same profile run of an application, starting at 1. Optional.

• desc: type: value [Cachegrind]
  This specifies various information for this dump. For some types, the semantic is defined, but any description type is allowed. Unknown types are ignored.

2.3 Description of Body Lines

(to be continued)