Secator docs
  • GETTING STARTED
    • Introduction
    • Installation
    • CLI Usage
    • Library usage
    • Configuration
    • Examples
      • 5 minutes secator session
  • RUNNER OPTIONS
    • Global options
    • Meta options
    • Input formats
    • Output options
  • IN-DEPTH
    • Philosophy & design
    • Distributed runs with Celery
    • Concepts
      • Output types
      • Proxies
      • Exporters
      • Runners
      • Drivers
      • Profiles
    • Deployment
  • For developers
    • Development setup
    • Writing tasks
      • Integrating an external command
        • Parsing JSON lines
        • Parsing raw standard output
        • Parsing output files
        • Example: integrating ls
        • Example: cat hunters
      • Integrate custom Python code [WIP]
      • Advanced options
    • Writing workflows
    • Writing scans [WIP]
Powered by GitBook
On this page
  • Setting the file path
  • Using a regular expression to get the file path

Was this helpful?

  1. For developers
  2. Writing tasks
  3. Integrating an external command

Parsing output files

... or how to integrate tools that save their output to a file.

If your tool supports output as individual files, you can easily integrate it with secator.

  • If you can set the output file path in advance, read Setting the file path.

  • If you cannot know, set, or guess the output file path in advance, read Using a regular expression to get the file path.


Setting the file path

Steps:

  • Set the file path before the command runs using the on_init hook

  • Use the on_cmd_done hook to read the file and yield secator output types.

Example:

Assume mytool outputs to a JSON file like:

mytool -u mytarget.com -format json -o result.json
...

and the corresponding JSON would look like:

[
  { "type": "url", "url": "http://mytarget.com", "status": 200 },
  { "type": "vulnerability", "target": "https://mytarget.com", "cve_id": "CVE-XXXX-XXXX", "name": "Bad vuln", "severity": "CRITICAL"},
]

An integration of mytool with secator would look like:

secator/tasks/mytool.py
from secator.decorators import task
from secator.runners import Command
from secator.output_types import Url, Tag, Vulnerability
from secator.serializers import RegexSerializer
from secator.tasks._categories import Vuln

OUTPUT_REGEX = r'\[INF\] JSON report will be saved to (?P<output_path>)'


@task()
class mytool(Command):
  cmd = '/home/osboxes/.local/bin/mytool'
  input_flag = '-u'
  json_flag = '-format json'
  output_types = [Url, Vulnerability]

  # Set the output file path
  @staticmethod
  def on_init(self):
    self.output_path = self.get_opt_value(OUTPUT_PATH)
    if not self.output_path:
      self.output_path = f'{self.reports_folder}/.outputs/{self.unique_name}.json'
      self.cmd += f' -o {self.output_path}'

  # When the command completes, load the JSON file and yield secator output types
  @staticmethod
  def on_cmd_done(self):
    with open(self.output_path, 'r') as f:
      results = f.read()
    for r in results:
      if r['type'] == 'url':
        yield Url(
          url=r['url'],
          status_code=r['status'],
        )
      elif r['type'] == 'vulnerability':
        yield Vulnerability(
          id=r['cve_id'],
          name=r['name'],
          matched_at=r['target'],
          severity=r['severity'].lower()
        )
  

Run it with secator:

$ secator x mytool mytarget.com

                         __            
   ________  _________ _/ /_____  _____
  / ___/ _ \/ ___/ __ `/ __/ __ \/ ___/
 (__  /  __/ /__/ /_/ / /_/ /_/ / /    
/____/\___/\___/\__,_/\__/\____/_/     v0.6.0

                        freelabz.com

No Celery worker alive.
/home/osboxes/.local/bin/mytool -u mytarget.com -format json -o <OUTPUT_FILE_PATH>
🔗 https://mytarget.com [200]
🚨 [Bad vuln 🡕] [critical] https://mytarget.com

Using a regular expression to get the file path

Steps:

  • Use the RegexSerializer to read and set the output file path.

  • Use the on_cmd_done hook to read the file and yield secator output types.

Example:

Assume mytool outputs to a JSON file like:

mytool -u mytarget.com -o json
[INF] JSON report will be saved to /path/to/custom/path.json
...

and the corresponding JSON would look like:

[
  { "type": "url", "url": "http://mytarget.com", "status": 200 },
  { "type": "vulnerability", "target": "https://mytarget.com", "cve_id": "CVE-XXXX-XXXX", "description": "Bad vuln" },
]

First we need to find a named regular expression that will match the filename.

Here is the one we came up with:

OUTPUT_REGEX = r'\[INF\] JSON report will be saved to (?P<output_path>)'

An integration of mytool with secator would look like:

secator/tasks/mytool.py
from secator.decorators import task
from secator.runners import Command
from secator.output_types import Url, Tag, Vulnerability
from secator.serializers import RegexSerializer
from secator.tasks._categories import Vuln

OUTPUT_REGEX = r'\[INF\] JSON report will be saved to (?P<output_path>)'


@task()
class mytool(Command):
  cmd = '/home/osboxes/.local/bin/mytool'
  input_flag = '-u'
  json_flag = '-o json'
  output_types = [Url, Vulnerability]

  # Override the default item loader (JSONSerializer) with the RegexSerializer
  item_loaders = [
    RegexSerializer(
      OUTPUT_REGEX,
      fields=['output_path']
    )
  ]

  # React to items loaded by the RegexSerializer, and set the output path
  @staticmethod
  def on_regex_loaded(self, item):
    self.output_path = item['output_path']
    return

  # When the command completes, load the JSON file and yield secator output types
  @staticmethod
  def on_cmd_done(self):
    with open(self.output_path, 'r') as f:
      results = f.read()
    for r in results:
      if r['type'] == 'url':
        yield Url(
          url=r['url'],
          status_code=r['status'],
        )
      elif r['type'] == 'vulnerability':
        yield Vulnerability(
          id=r['cve_id'],
          name=r['name'],
          matched_at=r['target'],
          severity=r['severity'].lower()
        )

Run it with secator:

$ secator x mytool mytarget.com

                         __            
   ________  _________ _/ /_____  _____
  / ___/ _ \/ ___/ __ `/ __/ __ \/ ___/
 (__  /  __/ /__/ /_/ / /_/ /_/ / /    
/____/\___/\___/\__,_/\__/\____/_/     v0.6.0

                        freelabz.com

No Celery worker alive.
/home/osboxes/.local/bin/mytool -u mytarget.com -o json
...
[INF] JSON report will be saved to /path/to/custom/path.json
🔗 https://mytarget.com [200]
🚨 [Bad vuln 🡕] [critical] https://mytarget.com
from secator.tasks import mytool
task = mytool('mytarget.com')
for item in task:
    print(item)  # this will output Url, Vulnerability, or Tag items.

PreviousParsing raw standard outputNextExample: integrating ls

Last updated 7 months ago

Was this helpful?