Getting Started¶
Before we start, we would like to introduce some basic concepts we use in yaramod so you can get familiar with them and know how to use them.
Yaramod as a library serves two main purposes:
Parsing of YARA rulesets into intermediate form which is friendly for inspection, analysis or even transformations
Creation of new YARA rulesets through code using declarative description of how your YARA rule should look like
These two together form a strong tooling for you to build more advanced systems, perform different kind of analyses over your rulesets, etc.
We have our own custom parser for YARA, so we don’t use the one in VirusTotal/yara, but we try to stay consistent as much as we can. Generally, you shouldn’t run into situation where YARA does parse your rule and yaramod does not (or vice versa) but this should be considered as bug and should be reported to us using issue tracker on GitHub.
It is also important to state that yaramod in no way does any kind of matching because it does not understand your ruleset in a sense what you want to match. Yaramod just knows what is inside of your rules and gives you an option to process it. Despite that, yaramod performs semantic checks where possible so you for example need to provide correct types of parameters to module function calls or you have to import a module in order to use it etc.
Modules¶
YARA is easily extensible with modules which provide you a way to call C functions from your YARA ruleset. We realize that modules are important and yaramod therefore supports all modules in upstream YARA.
We at Avast use YARA daily and sometimes there are things we would like to match in our YARA rules something that is not accessible to the outside world so we improve existing or develop our own modules and then supply yaramod with these modules as jsons. We will now describe how to use some custom modules.
First thing that needs to be done is to write each of the custom modules in a .json
file in a similar way the upstream modules are written in (see cuckoo module).
Then you have two options on how to supply the modules to yaramod depending on your preference:
Supply the directory, where the custom modules are stored, to
Yaramod
constructor with parameter modules directory as described in theYaramod instance
section. This means yaramod will load modules from specified directory instead of the directory, where the default upstream modules are specified.Supply the paths to the modules through environmental variable
YARAMOD_MODULE_SPEC_PATH
orYARAMOD_MODULE_SPEC_PATH_EXCLUSIVE
: SettingYARAMOD_MODULE_SPEC_PATH="<path to first module>:<path to second module>:<path to third module>"
means that you want to use the default YARA modules together with three aditional module specifications. Any number of paths separated by colon can be specified here. When theYARAMOD_MODULE_SPEC_PATH_EXCLUSIVE
environmental variable is set instead, yaramod will only consider the custom modules and no other modules will be available. Note, that when both variables are set yaramod throws an error.You can combine the two options and load modules from specified directory and additionaly some modules specified via
YARAMOD_MODULE_SPEC_PATH
.
When yaramod gets multiple different jsons describing the same module, it merges both specifications and builds module with functionality of boths specifications. We can merge structures, functions, add overloads of functions and more. An exception is thrown when there is type inconsistence or other conflict between the two specifications.
VirusTotal symbols¶
YARA allows you to define so called external variables which are values outside your YARA environment but they are constant across the whole YARA scan. You can reference them in your rule conditions to give an additional data to your rule so it can provide you with the matches you want.
These external variables are widely used on VirusTotal as you can see here. We realize this and we also realize that you might want to process rulesets that you are using in VirusTotal so we also provide these additional symbols but you can opt-out of this feature if you want.
Yaramod instance¶
The entrypoint of all yaramod is class Yaramod
which you should instantiate and keep it alive while you are doing anything with yaramod. Accessing internal representation
of YARA rules which were returned by Yaramod
is completely unsafe and can lead to crashes of your application if you do it after Yaramod
has been destructed. Creation of Yaramod
object
is performance heavy so you should keep the amount of instances low (ideally just one). Yaramod
itself is not thread-safe, so in parallel environment we would suggest
you to create one instance per thread or process.
Yaramod
accepts two optional parameters when creating it and that is import features and modules directory. The first option specify in what kind of rulesets you are interested in and you can choose from:
All current - This is the default option which provides you with both Avast-specific and VirusTotal-specific symbols.
Everything - This also includes deprecated functions which should no longer be used.
Basic - This represents that you are not interested in any additional symbols in your rules.
Avast - You are interested in basic and Avast-specific symbols.
VirusTotal - You are interested in basic and VirusTotal-specific symbols.
The second option enables you to supply custom modules other than YARA upstream modules. Simply enter the path to the directory where your modules are stored as json files.