Skip to content

Latest commit

 

History

History
206 lines (150 loc) · 6.26 KB

README.md

File metadata and controls

206 lines (150 loc) · 6.26 KB

AdGuard Filters Compiler

Filters compiler package is a tool for compiling ad blocking filters into a supported format. It is used in FiltersRegistry.

Usage

This package is suggested to be used with filters repository with directory structure presented in tests here.

The package could be run with the following command:

const whitelist = [1, 3];
const blacklist = [2];

const path = require('path');
const compiler = require("adguard-filters-compiler");

const filtersDir = path.join(__dirname, './filters');
const logPath = path.join(__dirname, './log.txt');
const reportPath = path.join(__dirname, './report.txt');

const platformsPath = path.join(__dirname, './platforms');

const customPlatformsConfig = {
    // Here you can redefine some of the platforms from platforms.json
    // or add new platforms if you need it.
    "MAC_V3": {
        "platform": "mac",
        "path": "mac_v3",
        "configuration": {
            "ignoreRuleHints": false,
            "removeRulePatterns": [
                "^\\/.*" // remove regex rules for some reason.
            ],
            "replacements": [
                {
                    "from": "regex",
                    "to": "repl"
                }
            ]
        },
        "defines": {
            "adguard": true,
            "adguard_app_mac": true
        }
    },
};

compiler.compile(filtersDir, logPath, reportPath, platformsPath, whitelist, blacklist, customPlatformsConfig);

Tests

yarn test

Development

In order to add support for new scriptlets and redirects, you should update @adguard/tsurlfilter with updated scriptlets.

For fixing scriptlets converting or validation you should update scriptlets.

Filters metadata

Description of the filters metadata is available in the FiltersRegistry repository.

@include directive and its options

The @include directive provides the ability to include content from the specified address.

Syntax:

@include <filepath> [<options>]

where:

  • <filepath> — required, URL or same origin relative file path to be included;

  • <options> — optional, a list of options separated by spaces. Available options:

    • /stripComments removes AdBlock-style syntax comments from the included file — lines which start with !;
    • /notOptimized adds a !+ NOT_OPTIMIZED hint to the rules;
    • /exclude="<filepath>" excludes from the included file rules listed in the exceptions file available by filepath;
    • /addModifiers="<modifiers>" adds the specified modifiers (string as is) to the rules in the included file. The addModifiers option can also work with the host-rule format files. In this case, host-file comments are to be replaced # by AdBlock-style syntax comments !;
    • /ignoreTrustLevel disables the check of the trust level of the included file. Allowed only for the same origin files.
    • /optimizeDomainBlockingRules remove redundant rules for domain blocking of the included file. Base rules with modifiers and rules of other format will be ignored. Comments are not checked separately and may not be relevant after optimization.

Important

The content of the included file is formatted by the options due to the order of their mention in the directive, except /ignoreTrustLevel.

Examples

  • Include a file with domains, add modifiers to the rules, exclude some rules, add a hint to the rules, and remove comments from the prepared rules:

    @include ../input.txt /addModifiers="script" /exclude="../exclusions.txt" /notOptimized /stripComments /optimizeDomainBlockingRules

    The order of execution of the options is as follows:

    1. @include ../input.txt: Includes the content of the file named input.txt from the parent directory.

      # comment
      example.com
      example.org
    2. /addModifiers="script": Adds the $script modifier to all rules in the included file.

      Result of adding modifiers:

      ! comment
      example.com$script
      example.org$script

      Used to restrict rules with modifiers when blocking the entire domain would result in a breakage. issue example

    3. /exclude="../exclusions.txt": Excludes rules listed in the exception list from the file named exclusions.txt, if they match.

      Due to the content of exclusions.txt:

      example.com$script
      example2.com$script

      Result of excluding:

      ! comment
      example.org$script

      Used to exclude problematic rules in the filter

    4. /notOptimized: Adds the !+ NOT_OPTIMIZED hint to the rules.

      Result of adding the hint:

      ! comment
      !+ NOT_OPTIMIZED
      example.org$script

      Used in cases where the filter is designed for mobile site layout and some rules may be removed, due to the lack of ability to collect statistics on mobile platforms.

    5. /stripComments: Removes comments in AdBlock style from the included file.

      !+ NOT_OPTIMIZED
      example.org$script
    6. /optimizeDomainBlockingRules: Remove only domain blocking redundant rules from the included file.

      Due to the optimization:

      ||example.com^
      ||sub.example.com^
      ||domain.com^
      ||test.com^$script

      Result of optimization:

      ||example.com^
      ||domain.com^
      ||test.com^$script
  • Ignore the trust level of the filter list (specified in the metadata) during the file including — include the file rules as is:

    @include ./input.txt /ignoreTrustLevel