SnipTest: Fuzzing Multi-Level Code Slices for Validating Vulnerabilities

Static analysis tools are widely used to detect potentially harmful bugs (i.e., vulnerabilities) early in the software development lifecycle [3, 42, 61, 53]. However, they often generate a large volume of warnings, many of which are false positives [31, 45, 47, 1]. In parallel, new code are frequently committed into the codebase, and these changes must be carefully reviewed to prevent regressions in production systems [15]. As a result, developers face a costly and error-prone triage process to determine which warnings or commits require urgent attention and which can be deprioritized.

A promising idea is dynamic validation—a family of techniques that attempt to confirm whether a suspicious code location actually results in failures at runtime. Among these, fuzzing is particularly intriguing due to its effectiveness in uncovering vulnerabilities [39, 14]. However, applying fuzzing to large codebases or individual warnings is challenging: writing tailored harnesses for each warning is labor-intensive; while general fuzzing often reach a coverage plateau before reaching the intended code location [8].

These limitations have led to the emergence of directed fuzzing, which is further categorized into distance-based [10, 12, 32] and slicing-based. In slicing-based fuzzing, a program slice (i.e., code closely related to the warning or commit) is isolated and fuzzed independently [44, 29, 21, 13]. Compared with distance-based fuzzing, slicing-based fuzzing starts at code locations that are much closer to the suspicious code, which significantly increases the chances of hitting the target. While one cannot get an end-to-end input that proves the existence of a bug, the under-constrained nature of a slice is especially useful for ruling out false bug reports, i.e., if a bug cannot be triggered with even less constraints on the inputs, it is more likely to be a false alarm. This is the core idea behind FuzzSlice [44].

However, current slicing-based techniques face a key limitation. Program slices are generated ahead and independent of fuzzing. More specifically, the scope of slices is often predetermined based on heuristics and does not adapt to fuzzing results. This implies that the fuzzer may either omit necessary context (if the slice is too small [44]) or introduce excessive complexity (if the slice is too large [10]). Furthermore, predetermined slices imply a limited set of execution paths or entry points, which can lead to false negatives by ignoring alternative call chains that may reach the same vulnerability  [19]. These constraints can limit their ability to expose exploitable vulnerabilities or correctly dismiss benign warnings.

To address these challenges, we introduce SnipTest, a compositional fuzzing framework that validates vulnerabilities through slices of increasing context. Instead of relying on a fixed set of slices, SnipTest organizes validation around slice levels. At Level 0, SnipTest generates a slice rooted at the function that contains the warning. At Level 1, it expands the analysis to immediate callers of that function and their dependencies. At Level 2, it further expands to higher-level callers, and so on. Critically, this leveled design is not merely about testing larger slices. The power lies in treating the trajectory of outcomes across levels as a diagnostic signal. A warning that triggers only in the most underconstrained slice (Level 0) is likely brittle and context‑sensitive—it may disappear once caller guards or initialization operations are reintroduced. In contrast, a warning that persists as we expand the calling context points to a systemic issue that warrants prioritization. By leveraging the sequence of outcomes rather than any single slice in isolation, SnipTest elevates slice‑based fuzzing from a filtering technique to reactive slicing for distinguishing benign anomalies from genuinely critical vulnerabilities. Figure 2 illustrates this process using a call graph in which the vulnerable function is progressively embedded in broader calling contexts.

Our work SnipTest improves on FuzzSlice [44] to support multi-level slicing—the idea of progressively expanding slices backward from a vulnerability. Multi-level slicing introduces new challenges such as handling an explosion of paths at higher levels, and avoiding redundant fuzzing across similar slices. SnipTest tackles these challenges through dedicated optimizations, time-bounded slicing, and a staged evaluation design that makes multi-level analysis tractable and meaningful (Sections V &  VI). SnipTest tests each bug across multiple slice levels to assess not just whether it can trigger a vulnerability, but how consistently it does so across all slices, providing a clearer indication of the bug likelihood to represent an exploitable vulnerability.

Throughout the paper, we use “validation” in a pragmatic sense, i.e., SnipTest produces execution evidence for warning triage in slice context, rather than proving full-program exploitability. A warning classified as a possible true positive indicates that SnipTest triggered the corresponding bug oracle within the analyzed sliced contexts, while a possible false positive indicates that the warning was reached without triggering the bug oracle in those contexts.

Our study contributes the following advances:

• •

  Multi-context warning triage. We introduce a warning-triage protocol that organizes executable slices by call-graph distance from a warning. SnipTest changes the unit of reasoning from one local function slice to a family of progressively constrained execution contexts. It aggregates observations within each level and across levels to determine whether the warning persists, disappears, or becomes unreachable as caller context is restored. To the best of our knowledge, SnipTest is the first work to dynamically adjust code slices based on fuzzing feedback.

• •

  Engineering support for scalable multi-level analysis. We introduce caching, pruning, staged stopping, parallel execution and wrapper-variable trimming to make the construction and fuzzing of multiple caller-rooted slices practical. These are engineering extensions around the low-level slice construction inherited from FuzzSlice.

• •

  Comprehensive evaluation. We evaluate SnipTest across four key dimensions: effectiveness, fidelity, efficiency in detecting true vulnerabilities and false alarms. Our work is comprehensive due to the indepth discussion of true positives, true negatives, false positives and false negatives - rigor not commonly found in prior work [44, 30, 49]. We also provide direct comparison with the directed fuzzer AFLGo [10] and the LLM-based approach LLM4SA [55]. SnipTest achieves a 5.5-10.6x speedup in fuzzing time over seeded or unseeded AFL-Go. Additionally, SnipTest outperforms LLM4SA, with an F1 score of 0.791 versus 0.360.

• •

  Practical vulnerability detection. We demonstrate the practicality of SnipTest by detecting 3 real world vulnerabilities in 2 Open Source projects [48, 46, 17] leading to disclosure of CVE-2025-11964 [22].

• •

  Artifact & Reproducibility. Our replication package and datasets are publicly available.

In this section, we provide background on coverage-guided fuzzers and directed fuzzers.

Coverage-guided fuzzers. Fuzzing is an automated software testing technique that generates program inputs to expose crashes, bugs, and security vulnerabilities. Traditional fuzzers operate in an undirected manner: they aim to maximize overall code coverage or execution diversity without prioritizing any specific program locations. Popular coverage-guided fuzzers, such as AFL and libFuzzer [2, 40], fall into this category and have been widely successful in discovering shallow and medium-depth bugs.

Directed fuzzers. Directed fuzzing extends this model by biasing input generation toward specific targets, such as known vulnerable code locations, patches, or warnings reported by static analysis tools. Instead of maximizing global coverage, directed fuzzers such as AFLGo and HawkEye [10, 23], guide execution toward target locations using metrics such as control-flow distance or reachability. This approach is particularly effective for deep or hard-to-reach bugs but typically requires accurate target specification and may incur additional overhead in analysis and guidance.

Our goal is to validate whether suspicious code locations, such as those emerged by static analysis tools or recently changed, correspond to actual vulnerabilities. For instance, if a tool reports a potential memory leak or buffer overflow, we aim to determine whether the issue is a true bug or a false alarm. Manually inspecting each warning or fuzzing the entire program in response to each one is inefficient and often ineffective [9, 8].

To illustrate our approach, we present a motivating example from the binutils project [7]. Listing 1 shows simplified C code involving three functions: the function btrace_alloc performs heap allocation for a struct and its fields, btrace_clear partially deallocates the structure, and use_btrace orchestrates the full flow.

When a static analysis tool such as Infer [24] analyzes this code, it reports a memory leak associated with the field p in the btrace_alloc function. Validating this warning using whole-program fuzzing, such as fuzzing the entire binutils project, is slow, often requiring days of execution and project-specific fuzzing dictionaries, and may still fail to trigger the vulnerability.

SnipTest adopts a more targeted strategy. It begins by generating and fuzzing a minimal code slice containing the warning location, which we refer to as Level 0. In this example, Level 0 includes only the function btrace_alloc. When fuzzed in isolation, a bug oracle e.g. LeakSanitizer [34] reports memory leaks for all three heap-allocated components: p, q, and X.

To improve diagnostic precision, SnipTest incrementally expands the slice. At Level 1, SnipTest includes the caller use_btrace and its dependencies, such as btrace_clear. This broader slice enables both allocation and deallocation logic to be exercised. Fuzzing at this level suppresses the leak reports for q and X, as valid deallocation paths are now included. However, the warning for p persists, since p is never freed. At higher levels, we would still expect p to be leaked.

This example highlights the core insight behind SnipTest: by observing how warnings behave across incrementally expanded slice levels, we can distinguish between true bugs and false alarms. If a warning consistently triggers across multiple levels, it is likely signaling a true vulnerability. If the code is executed but the warning does not manifest, it is likely a false alarm. That is, iteratively growing the code slice around the warning and checking if the warning persists, SnipTest can offer a lightweight yet semantically grounded way to prioritize warnings without requiring full-program fuzzing.

Program slicing is well-studied, with most work focusing on uncompiled backward slices from entry points to vulnerabilities [37, 6, 33]. However, these slices are often overly large for arbitrary code points [4, 51, 5, 26]. In contrast, SnipTest operates at the function level, terminating backward slices at different functions, and finally compiling and linking small slices to enable quick exploration.

A common technique for dynamically validating vulnerabilities is directed fuzzing. Prior work has proposed various strategies for directing fuzzers toward specific program regions [36, 23, 57, 12, 11, 27], primarily by mutating inputs to increase the likelihood of reaching a target location. Therefore directed fuzzers can be used to verify static analysis warnings. The key distinction between SnipTest and directed fuzzing lies in their entry points: while directed fuzzers typically begin at the main function and explore the entire program, SnipTest generates small slices around warning locations and confines fuzzing within these slices. Input mutation strategies from directed fuzzing are orthogonal to SnipTest and could be integrated into its fuzzing engine.

FuzzSlice is the closest implementation and methodological predecessor to SnipTest. Given a target function, FuzzSlice resolves dependencies, extracts a compilable function-rooted slice, generates a fuzzing wrapper, and tests the resulting binary. Its unit of reasoning is a single underconstrained local slice, and its primary use case is false-positive filtering. SnipTest reuses this slice-materialization operation—it does not introduce a new low-level slicing algorithm. Rather, its contribution lies in the surrounding framework: selecting multiple caller roots via static call-graph analysis, grouping slices by caller distance, testing multiple contexts per level, aggregating observations within and across levels, and applying staged stopping and scalability optimizations. Level 0 is the closest comparable setting to FuzzSlice, as both analyze a slice rooted at the warning-containing function. However, SnipTest is not identical even at Level 0, owing to wrapper-generation improvements. More importantly, Level 0 is only the initial step in a multi-level warning-triage process. While FuzzSlice focuses on underconstrained exploration of fine-grained slices primarily for false-positive filtering, SnipTest supports adjustable slice sizes, enabling improved precision and more effective vulnerability detection. Beyond that, SnipTest incorporates static analysis to automatically discover ancestor functions, expands slices based on fuzzing outcomes, and introduces multiple optimizations to classify warning behavior across progressively larger code contexts as benign, vulnerable, or unreachable.

Runtime bug oracles such as AddressSanitizer, LeakSanitizer, UndefinedBehaviourSanitizer and MemorySanitizer [50, 35, 52, 43] are used by SnipTest to detect several issues (e.g., out-of-bounds access, null pointer dereference, timeouts, double free), but other oracles—such as developer-provided assertions or even LLM-based assertions—can be integrated at runtime without modifying the framework. The design of effective oracles is complementary to, and independent of, SnipTest itself. As more expressive sanitizers or assertions become available, SnipTest (like any fuzzing system) can naturally detect a broader class of bugs, including logic errors.

Other approaches for identifying actionable static-analysis warnings include machine-learning–based techniques and, more recently, LLM-based approaches. Machine-learning–based methods train classifiers to distinguish vulnerable from non-vulnerable code using features extracted from static-analysis outputs, such as warning types, code metrics, or data-flow characteristics, and they require large, high-quality labeled datasets [20, 58, 59]. LLM-based approaches, in contrast, leverage pretrained language models to reason about code semantics and warning context, reducing the need for handcrafted features but still relying heavily on the quality and availability of static-analysis information or prompt context [56, 38]. These approaches differ fundamentally from SnipTest; they operate at the prediction level without executing the program.

SnipTest aims to validate potential vulnerabilities reported by static analysis tools or recently changed code by generating and testing executable code slices that include the reported vulnerability across various program contexts. This is done through four key steps: (1) Construct a static function-level call graph; (2) Identify the function enclosing the vulnerability; (3) Generate and compile slices of increasing context; (4) Fuzz the resulting binaries and classify the vulnerability. We describe each step below.

Each level adds more calling context, allowing us to test whether the vulnerability warning depends on external inputs or conditions. If a vulnerability is triggered at Level N, further expansion is necessary as higher levels may reveal hidden preconditions that prevent the vulnerability from being exploited. If the suspected code location is reached sufficiently but no vulnerability is triggered, further expansion may be unnecessary [44].

Figure 2 shows a call graph where main calls both A and D; A calls B, which in turn calls C and D. The vulnerable function is C. The slice expands progressively, i.e., in Level 0, only function C is included (minimal slice). Level 1 includes functions B, C, and D in one slice, as B has a maximum calling distance of 1 from C and B’s dependencies are C and D. Level 2 further includes function A in a single slice, with a maximum calling distance of 2 from C. The slice includes A and its recursive dependencies B, C, D. Depending on the callgraph, there can be multiple slices at a given level.

For each slice level, we identify all functions whose distance from the vulnerable function matches that level. In Figure 2, for instance, only function C (the vulnerable function) is sliced at Level 0. At Level 1, we slice function B, which is one level up from C, and its dependencies. At Level 2, function A is also included.

This step finally produces multiple binaries, each containing a code slice with a defined amount of context.

At this stage, multiple code slices have been compiled and linked, each corresponding to a different execution path that leads to the suspected function. We iteratively perform fuzzing, starting from binaries at Level 0, followed by those at Level 1, and so forth. SnipTest uses AFL++ [16] to fuzz the binaries and collects coverage information using llvm-coverage[40] in order to track execution frequencies of each line, alongside the results of runtime bug oracles (e.g., ASAN[50] and UBSAN [52]) to detect vulnerabilities.

The challenge at this stage is to consolidate the fuzzing results across different execution paths and slice levels to make a final classification of the vulnerability warning. We perform classification at three stages:

1. (a)

   Triggered: The bug oracle detects a vulnerability at the warning location, confirming that the warning has been triggered in the code slice.

2. (b)

   Reached: The warning location is executed, but no bug is detected by the bug oracle, suggesting that while the path is feasible, it does not necessarily lead to an actual vulnerability.

3. (c)

   Not Reachable: The warning location is never executed during fuzzing, indicating that no execution path in the slice reaches it.

1. (a)

   Vulnerable (V): If at least one binary at the current slice level is triggered, then a vulnerable execution path has been identified. This suggests that the warning location is likely a true positive.

2. (b)

   Benign (B): If no slice at the current level is triggered, but at least one slice is reached, it indicates that execution paths to the vulnerability warning exist, but no vulnerabilities have been exploited. This suggests that the warning is likely a false positive, as an extensive search fails to identify an offending input that triggers the bug.

3. (c)

   Not Reachable (NR): If all binaries at the current level classify the warning as Not Reachable, no feasible execution path leads to the warning at this level. In such cases, the warning location remains undetermined, and we cannot make a conclusive classification at this slice level.

1. (a)

   Possible True Positive (PTP): If the warning is consistently labeled as “Vulnerable” at all slice levels, it is classified as a PTP, indicating a high likelihood of an actual vulnerability.

2. (b)

   Possible False Positive (PFP): If the warning is reached through “Benign” paths at lower slice levels and remains benign or “Not Reachable” at higher levels, it is classified as a PFP. This reflects our validated assumption (Section VII) that warnings benign at shallow slices are unlikely to become vulnerable in deeper ones [44].

3. (c)

   Not Reachable (NR): If the warning remains “Not Reachable” at all slice levels, it means that conclusions cannot be drawn about the validity of the warning, as it was never reached during fuzzing.

The final classification rule is a heuristic evidence-aggregation strategy rather than a formal proof of vulnerability or benignness. Its purpose is to support warning triage by summarizing how a warning behaves across increasingly broader sliced execution contexts. Consistent triggering across slice levels increases confidence that the warning should be prioritized, while reaching the warning without triggering the bug oracle provides evidence that the warning may be benign under the explored contexts. Accordingly, we use cautious labels—Possible True Positive, Possible False Positive, and Not Reachable—to reflect the evidential nature of the classification.

While SnipTest systematically tests vulnerability warnings through iterative slicing and fuzzing, practical challenges may arise. Specifically, as slice levels grow, the number of execution paths can increase exponentially, leading to excessive slice creation and redundant fuzzing. Since each execution path requires a separate binary for analysis, inefficient slicing and fuzzing can significantly inflate computational costs.

To address these challenges, SnipTest proposes a set of optimizations that fall into two categories: (1) Slice Creation Optimizations; (2) Slice Fuzzing Optimizations. Below, we describe each set of optimizations in detail.

We organize the evaluation of SnipTest around the following research questions:

• •

  RQ1: How effective is SnipTest at validating true vulnerabilities across slice levels?

• •

  RQ2: How effective is SnipTest at filtering out false alarms across slice levels?

• •

  RQ3: How accurately does SnipTest reproduce the actual execution paths of true vulnerabilities in stack traces?

• •

  RQ4: How efficient is SnipTest with its slice-level optimizations?

• •

  RQ5: How does SnipTest compare with directed fuzzers in efficiency and target reachability?

• •

  RQ6: How does SnipTest compare with LLM based approaches for classifying vulnerabilities?

• •

  RQ7: How does SnipTest compare with slice based approach FuzzSlice?

To answer these questions, we first describe our evaluation setup in Section VII-A, including dataset construction and SnipTest configuration. We then report results for each RQ. in Sections VII-B–VII-H.

Using the static analysis tool Flawfinder [18], we generated 2,328 warnings for Vim and 1,284 for libpcap. From these, we selected warnings corresponding to high-impact vulnerabilities, such as CWE-120 (buffer overflow) and CWE-476 (null pointer dereference), which were not covered by native tests. This process yielded 24 warnings in Vim and 9 in libpcap, which we then attempted to validate using SnipTest.

We fuzzed the associated multi-level code slices (levels 0–2) containing the sampled warnings. Out of the 33 candidate warnings, only four produced consistent crashes across all slice levels. For these, we submitted issues and pull requests to the affected projects. Within a week, this effort led to the confirmation of three new vulnerabilities across both projects: two buffer overruns and one null pointer dereference [48, 46, 17]. The fourth case, an out-of-bounds write in Vim, was disputed by developers, who argued in an email thread that the behavior was intentional and not a bug. We discuss each identified vulnerability and its fix below:

1. 1.

   Buffer overflow in libpcap [17]. Libpcap is a portable API for low-level network monitoring and traffic analysis across Unix-like systems and Windows (via WinPcap). A heap buffer overflow was identified in the function utf_16le_to_utf_8_truncated, which is responsible for converting Windows error messages from UTF-16 to UTF-8. During this conversion, the function incorrectly counts the number of characters written when determining how much data to truncate, resulting in a heap buffer overwritten at the end of the destination buffer. This vulnerability was assigned CVE-2025-11964 [22]. The developer’s fix corrects the character counting logic used during conversion. The developer released a new version of the software patching the vulnerability and appreciating the authors for spotting the issue in the release notes [22].

2. 2.

   Buffer overflow in Vim [48]. Vim is a highly configurable text editor designed for efficient text creation and modification. A buffer overflow was discovered in Vim within the function cs_pathcomponents, which truncates file paths based on a user-supplied integer value (e.g., displaying only the last N directories of a path). The truncation logic contains an error in string pointer manipulation—specifically, a double decrement—leading to a buffer underflow. The issue was fixed by correcting the pointer logic to perform a single decrement instead of two.

3. 3.

   Null dereference in Vim [46]. A null pointer dereference was identified in Vim in the function cs_find_common, which is executed when the find command is invoked within cscope. During execution, the function attempts to create a temporary file. If this operation fails (e.g., due to insufficient disk space, filename conflicts, or inadequate permissions), the resulting null pointer may still be accessed later in the function, causing a crash. The fix applied for the bug ensures that the function exits early if temporary file creation fails, preventing dereference of a null pointer.

The remaining warnings either did not produce crashes at all slice levels or were not reachable (categorized as PFP or NR). While our analysis does not constitute a full-scale evaluation of several static analysis warnings, we intended to demonstrate the practical utility of SnipTest in identifying real vulnerabilities in widely-used open-source software.

We now discuss an exhaustive list of factors in the SnipTest framework that can create false positives and false negatives. We also discuss potential mitigation strategies.

1. 1.

   Incomplete minimal context for bugs (false positives only). A reported bug may be classified as a true vulnerability or a false alarm based on a guard condition that resides several functions earlier in the call hierarchy. In our current implementation, SnipTest limits analysis to at most three slice levels. Extending the analysis to additional slice levels could incorporate such transitive guard conditions and help avoid misclassifying false alarms as true vulnerabilities.

2. 2.

   Insufficient fuzzing time relative to slice size (false negatives or unreached code). In some cases, the allocated fuzzing time is insufficient to adequately explore larger code slices. In our evaluation, the time budget is 5/7.5/10 minutes for Levels 0, 1 and 2. These limits are empirically chosen and may be inadequate for certain complex slices. Moreover, large test cases (e.g., functions with many or large arguments) and inefficient mutation strategies can further hinder exploration. Such limitations may result in unreached code or false negatives - where reachable bugs are not exploited within the allotted time. For example, Table XII shows 5 repetitions of SnipTest on our dataset. We observed 2 cases in libxml2 (dataset-TP) crashing at all slice levels on some iterations and not being reached in others. However in dataset-FP, the same cases can be reached at higher slice levels, classifying them as benign.

3. 3.

   Function pointers, file I/O, and runtime environment variables (false positives, false negatives or unreached code). SnipTest currently assigns function pointers to NULL and does not randomly bind them to functions within the slice. In addition, it does not model file-based I/O or runtime environment variables that influence program behavior. These limitations can lead to both false positives, false negatives or unreached code. Addressing these aspects is left for future work.

We presented SnipTest, an execution-based warning-triage framework built atop the FuzzSlice function-slice compilation primitive. SnipTest organizes slices into progressively broader context levels and aggregates warning behavior both within and across these levels. This methodology yields evidence about whether a warning persists as caller-imposed constraints are gradually restored. In our benchmark, SnipTest produced Possible True Positive evidence for 53 of 97 confirmed vulnerabilities and Possible False Positive evidence for 54 of 97 confirmed false alarms when analyzing across three slice levels. It also generated Possible True Positive evidence for 28 false alarms, underscoring that insufficient context remains an important limitation. SnipTest should therefore be viewed as a prioritization aid that complements developer review. Our evaluation further demonstrates that sliced execution can substantially reduce fuzzing time and expose previously unknown defects, as evidenced by our disclosure of CVE-2025-11964. These results support multi-level sliced fuzzing as a practical approach for gathering warning-triage evidence—rather than as definitive proof of vulnerability or benignness.