Set DexTrace as the default core library - #967
Open
haeter525 wants to merge 10 commits into
Open
Conversation
Classes referenced but not defined in the app's DEX (e.g. NetworkInfo, ConnectivityManager) had no entry in superclass_relationships, so find_api_usage()'s subtype walk dead-ended before reaching Ljava/lang/Object;. Rules pairing such a class with a generic Object.equals call as the second API could never reach full combination/sequence confidence. Every Java class ultimately extends Object, so default any class with no DEX-derived hierarchy entry to that instead of a lookup dead end. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JB4p3nzcxeJ16U7WQMEh3Q
- quark/cli.py, quark/core/quark.py, quark/report.py: default core_library changed from androguard to dextrace. quark/forensic/ forensic.py intentionally left on androguard -- it has no dextrace branch and DexTraceImp.get_strings() is a stub returning an empty set, so switching its default would silently break string extraction rather than just swap backends. - setup.py: move rzpipe and r2pipe==1.8.0 out of install_requires into new 'rizin' / 'radare2' extras. androguard and dextrace stay required. Shuriken was already optional (never in install_requires). - quark/core/axmlreader/__init__.py: the module-level 'import rzpipe' / 'import r2pipe' transitively broke every backend, since BaseApkinfo (which all backends inherit) imports axmlreader.python, which runs axmlreader/__init__.py as a package side effect. Verified empirically that BaseApkinfo failed to import with rzpipe/r2pipe unavailable before this fix. Moved both imports into the rizin/radare2 branches of AxmlReader.__init__, guarded with try/except raising AxmlException (the file's own local exception -- can't import the new shared exception here without a circular import back through baseapkinfo.py -> axmlreader.python -> axmlreader/__init__.py). - quark/core/rzapkinfo.py, quark/core/r2apkinfo.py: same guard pattern already used by ShurikenImp (try/except ModuleNotFoundError around the top-level import, check-and-raise moved into __init__). Now raise the new shared CoreLibraryUnavailable exception. - quark/core/interface/baseapkinfo.py: add CoreLibraryUnavailable, a shared exception for 'this --core-library backend's dependency isn't installed'. ShurikenImp now raises this too instead of a bare Exception. - quark/cli.py: wrap both Quark(...)/ParallelQuark(...) construction sites in try/except CoreLibraryUnavailable, printing a clean one-line message via print_error() and exiting 1, instead of letting a raw traceback reach the user (previously true even for Shuriken's existing 'not installed' message). - tests/core/test_quark.py, tests/agent/test_agentTools.py: pin the three Quark(...) call sites that relied on the implicit default to core_library="androguard" explicitly, since these fixtures test Quark's evaluation logic against Androguard-specific behavior, not 'whatever the current default backend is'. Verified: BaseApkinfo, quark.core.quark, RizinImp, and R2Imp all import/construct correctly with rzpipe/r2pipe simulated missing (RizinImp/R2Imp raise CoreLibraryUnavailable with install instructions instead of ModuleNotFoundError at import time). CLI with no --core-library flag now runs dextrace by default end-to-end; --core-library androguard still works explicitly. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JB4p3nzcxeJ16U7WQMEh3Q
install.rst's --core-library list was already stale (missing dextrace before this change). Add it, mark it as the default, and document pip install quark-engine[rizin]/[radare2] for the two backends that moved out of install_requires. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JB4p3nzcxeJ16U7WQMEh3Q
- add _register_no_caller_callee_methods: walk all declared methods up front so a concrete method with no outgoing calls and no visible caller in-DEX (e.g. interface-callback overrides like HostnameVerifier.verify()) still lands in all_methods/android_apis/ custom_methods, instead of only being discoverable via extract_api_calls (invoke targets) or _register_abstract_methods (code_off==0 declarations) - wire extract_declared_methods and extract_strings into imports - implement get_strings() via extract_strings instead of returning an empty set
- add debug logger gated on config.DEBUG - sort per-caller calls by real invoke offset when all present; fall back to DexTrace's original call order (with a warning) when any offset is missing, instead of a stable-sort tiebreak that guessed order - propagate the real offset into _calls_by_caller and _calls_by_caller_sig instead of always using enumerate() index
Drop "_from_cache" suffix — implementation detail, not part of the name's meaning; caller doesn't care about caching.
- strip quotes and unescape smali string literals (const-string etc keep quotes; androguard's operand is the raw unquoted string) - parse const/const-wide literal operands (decimal, 0x-hex, optional 'L' wide suffix) into int to match androguard's int operand type
Method.get_upper_methods walked apkinfo.analysis.get_class_analysis, an androguard-only API; DexTrace backend has no analysis object. Derive class hierarchy from apkinfo.superclass_relationships instead, which both backends provide. superclass_relationships mixes extends+implements edges; first parent is treated as the superclass edge since rules/backends don't distinguish the two either.
- swap Dict/List/Set/Tuple/Optional[X] for dict/list/set/tuple/X | None builtin generics (PEP 585, PEP 604) throughout - move Generator/Iterable imports to collections.abc - drop empty parens on functools.lru_cache decorators - drop unused second Generator type param (yields only)
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #967 +/- ##
=======================================
Coverage 78.48% 78.48%
=======================================
Files 84 84
Lines 7692 7692
=======================================
Hits 6037 6037
Misses 1655 1655
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Changes Quark's default
--core-libraryfromandroguardtodextrace, per the comparison results in #966.Also makes
rizin/radare2optional extras instead of hard dependencies.Adds a clear error message when a selected backend is unavailable.
Key Changes
--core-libraryvalue (CLI andQuark.__init__) todextrace.CoreLibraryUnavailableexception inbaseapkinfo.py.cli.pynow catches it and prints a clean error instead of a raw traceback.rzpipe/r2pipefrom core install requirements into new extras:pip install "quark-engine[rizin]"pip install "quark-engine[radare2]"quark/core/quark.py.returnon the first(first_api, second_api)pair with no mutual parent function aborted the whole rule check.continue, so remaining candidate pairs are still tried.PYTHONHASHSEED-dependent confidence-score flakiness.docs/source/install.rstfor the new default and extras.r2apkinfo.py,rzapkinfo.py,shurikenapkinfo.py,dextraceapkinfo.py,axmlreader/__init__.py,script/__init__.py,report.py.Motivation and Context
Closes part of #966.
A comparison between DexTrace and Androguard across 470 comparable samples from the malware-family corpus showed a 99.6% exact confidence-score match (468/470).
The 2 remaining mismatches trace to a known Androguard adapter bug, not to DexTrace:
Blocked on: DexTrace#19.
dextraceversion pin insetup.pyneeds bumping once that's out — until then CI here won't pass.How Has This Been Tested?
Checklist
dextraceversion pin insetup.py