Semantic Designs: CheckPointer compared to other safety checking tools

CheckPointer compared to other Memory Safety checking tools

The SD CheckPointer tool finds memory management errors in programs with explicit allocation, deallocation and pointer dereferencing operations. Here we compare this tool with other tools that accomplish related analyses. For each tool, the union of Errors Detected and Errors Missed are intended to cover all possible memory safety errors to enable effective tool comparisons.

The table should make it clear SafetyCheck is stronger than all the dynamic checking tools, and will detect any statically-detectable error that occurs in some actual execution run.

Tool NameLanguage
Analyzed
Analysis
Type
Errors
Detected
Errors
Missed
Status/
Availability
CheckPointerC (ANSI, MS, GNU)
C99 (ANSI, GNU)
Dynamic
using
compile-time instrumentation
All runtime memory safety access errors
including outside globals, locals, heap, strings, arrays or struct elements.
Memory leaks
Statically-detectable errors in un-executed codeCommercial
Windows
Linux (under Wine)
CCuredANSI C
with restrictions;
structs containing pointers change layout breaking compatibility
Dynamic
using
compile-time instrumentation
Runtime access errors outside globals and heap-allocated storage;
dangling pointer errors
Errors in structs whose layout changes due to "fat" pointers;
statically-detectable errors in un-executed code
Research
Linux?
SoftBoundClang (ANSI?) CDynamic
using
compile-time instrumentation
Runtime access errors
including outside globals, locals, heap, strings, arrays or structs
Temporal errors (double free);
errors involving C library block-memory moves (e.g. memmove);
statically-detectable errors in un-executed code
Research
Windows, Linux
MSCCANSI? CDynamic
using
compile-time instrumentation
Runtime access errors
including outside globals, locals, heap, and temporal errros (e.g., double free)
Problems with bad casts on dynamically allocated blocks
statically-detectable errors in un-executed code
Research
Only applied to small programs
Linux?
SafeCodeANSI? CDynamic
using code transformations
Prevents, not detects, memory access faults for dangling pointersStatically-detectable errors in un-executed codeResearch/Linux?
Clang variant?
Insure++ANSI? CDynamic
using
compile-time instrumentation
Statically detection of some uninitialized pointers
Runtime access errors
including outside globals, locals, heap;
memory leaks
Erroneous access to recycled heap memory;
statically-detectable errors in un-executed code
Commercial
Windows?, Linux, AIX
Bug
Fighter
C (GNU, MS)Dynamic
using
compile-time instrumentation
Most runtime array subscript errors;
null pointer dereferences
Errors for certain array access syntax;
other pointer-related errors;
some array index errors;
statically-detectable errors in un-executed code
Breaks some larger programs
Freeware
Windows
Bounds
Checker
Windows C, C++Dynamic
using
compile-time instrumentation
Runtime array subscript errorsErrors for certain array access syntax;
all pointer-related errors;
statically-detectable errors in un-executed code
Commercial
Windows
PurifyObject code:
several machine architectures; not Visual Studio C++
Dynamic
using load? time instrumentation
Runtime accesses outside global, stack or heap-allocated storage;
accesses outside program, stack or data segments;
free bad pointer;
double free;
memory leaks
Runtime access errors into reused heap storage;
access errors to local variables, or within a struct;
statically-detectable errors in un-executed code
Commercial
Windows, Linux, Solaris
Valgrindx86 object codeDynamic
using
runtime traps
Runtime access outside of heap allocated variables;
accesses outside of program, data or stack area
Accesses outside of global or local variables;
accesses outside of elements of structs;
statically-detectable errors in un-executed code
Open Source
Linux
DieHardANSI(?) CDynamic
replacing malloc
Statistically, most runtime accesses outside heap-allocated variablesRuntime accesses outside globals, locals or struct elements;
statically-detectable errors in un-executed code
Research
Linux?
Electric
Fence
Object code:
x86 instruction set
Dynamic
using load? time instrumentation
Runtime accesses outside global, stack or heap-allocated storage;
accesses outside program, stack or data segments;
free bad pointer;
double free;
memory leaks
Runtime access errors into reused heap storage;
access errors to local variables, or within a struct;
statically-detectable errors in un-executed code
Freeware?
Linux?
glibc
MALLOC
_CHECK_
GNU CDynamic
using malloc-internal checks
Runtime erroneous free operations
double free
Runtime accesses outside of heap-allocated variables
statically-detectable errors in un-executed code
Open Source
Windows, Linux
PC-LintANSI, MS, GNU CStatic
using heuristic flow analysis
Some pointer access errors;
many false positives
Runtime access errorsCommercial
Windows, Linux
BlastANSI(?) CStaticAll CCured-detectable problems;
some specified temporal safety properties
Any unspecified temporal propertyResearch
Linux?
CycloneCyclone: Type-safe variant of CDynamicNull dereference;
uninitialized pointer dereferences;
dangling pointers
Statically-detectable errors in un-executed codeResearch
Linux?
Prefix/
Prefast
Visual CStaticMany memory safety errors; problems with false positivesRuntime access errors not detectable by static analysisCommercial
Windows
Coverity
Prevent
C, C++, JavaStaticMany memory safety errors; problems with false positivesRuntime access errors not detectable by static analysisCommercial
Windows, Linux
FortifyC, C++, JavaStaticMany memory safety errors; problems with false positivesRuntime access errors not detectable by static analysisCommercial
Windows, Linux

We believe we have characterized these other tools fairly. If you think a tool is missing, or a description is inaccurate, or could be improved, please send a note to webmaster@semdesigns.com

For more information: info@semanticdesigns.com    Follow us at Twitter: @SemanticDesigns

Memory Safety CheckDMS
Comparison To Similar Tools