Secator docs
  • GETTING STARTED
    • Introduction
    • Installation
    • CLI Usage
    • Library usage
    • Configuration
    • Examples
      • 5 minutes secator session
  • RUNNER OPTIONS
    • Global options
    • Meta options
    • Input formats
    • Output options
  • IN-DEPTH
    • Philosophy & design
    • Distributed runs with Celery
    • Concepts
      • Output types
      • Proxies
      • Exporters
      • Runners
      • Drivers
      • Profiles
    • Deployment
  • For developers
    • Development setup
    • Writing tasks
      • Integrating an external command
        • Parsing JSON lines
        • Parsing raw standard output
        • Parsing output files
        • Example: integrating ls
        • Example: cat hunters
      • Integrate custom Python code [WIP]
      • Advanced options
    • Writing workflows
    • Writing scans [WIP]
Powered by GitBook
On this page
  • One-to-one mapping
  • One-to-many

Was this helpful?

  1. For developers
  2. Writing tasks
  3. Integrating an external command

Parsing JSON lines

... or how to integrate tools that output JSON lines.

If your tool outputs JSON lines, it's very easy to integrate it with secator.

Based on how your tool's output maps will map to secator output types, read:

  • One-to-one mapping (1 JSON line = 1 secator output type).

  • One-to-many (1 JSON line = manysecator output types).


One-to-one mapping

Steps:

  • Use the JSONSerializer item loader.

  • Add an output_map to map your tool's output to one of secator's Output types.

Example:

For instance, mytool outputs JSON lines when we run it like:

mytool -jsonl -u mytarget.com
{"url": "https://mytarget.com/api", "status": 200, {"details": {"ct": "application/json"}}
{"url": "https://mytarget.com/api/metrics", "status": 403, "details": {}}

An integration of mytool with secator would look like:

secator/tasks/mytool.py
from secator.decorators import task
from secator.runners import Command
from secator.output_types import Url
from secator.serializers import JSONSerializer
from secator.definitions import URL, STATUS_CODE, CONTENT_TYPE


@task()
class mytool(Command):
  input_flag = '-u'
  json_flag = '-jsonl'
  output_types = [Url]
  item_loaders = [JSONSerializer()]
  output_map = {
    Url: {
     URL: 'url',
     STATUS_CODE: 'status',
     CONTENT_TYPE: lambda x: x['details'].get('ct', '')
    }
  }

Steps taken:

  • Import the desired runner (Command).

  • Import the desired output type(s) and the field names that we want to map.

  • Set input_flag to mytool's input flag -u.

  • Set json_flag to mytool's JSON line flag jsonl.

That's it ! You can now run mytool with secator and enjoy all the features it brings on top of it:

secator x mytool mytarget.com
                         __            
   ________  _________ _/ /_____  _____
  / ___/ _ \/ ___/ __ `/ __/ __ \/ ___/
 (__  /  __/ /__/ /_/ / /_/ /_/ / /    
/____/\___/\___/\__,_/\__/\____/_/     v0.6.0

                        freelabz.com
mytool -u mytarget.com -jsonl
🔗 https://mytarget.com/api [200] [application/json]
🔗 https://mytarget.com/api/metrics [403]
from secator.tasks import mytool

task = mytool('mytarget.com')
for item in task:
    print(item)  # item is now a secator output type like Vulnerability or Port

One-to-many

Steps:

  • Add a static method on_json_loaded to hook onto the JSON Serializer output.

  • Modify the data and yield secator Output types.

Example:

For instance, if mytool outputs urls in batch:

mytool -jsonl -u mytarget.com
{"urls": [{"url": "https://mytarget.com/api", "status": 200, "content-type": "application/json"}, {"url": "https://mytarget.com/api/metrics", "status": 403, "content-type": "application/text"}]

An integration of mytool with secator would look like:

secator/tasks/mytool.py
from secator.decorators import task
from secator.runners import Command
from secator.output_types import Url
from secator.definitions import URL, STATUS_CODE, CONTENT_TYPE


@task()
class mytool(Command):
  input_flag = '-u'
  json_flag = '-jsonl'
  output_types = [Url]

  @staticmethod
  def on_json_loaded(self, data):
      for item in data['urls']:
          yield Url(
            URL: item['url'],
            STATUS_CODE: item['status'],
            CONTENT_TYPE: item.get('content-type', '')
          )

That's it ! You can now run mytool with secator and enjoy all the features it brings on top of it:

secator x mytool mytarget.com
                         __            
   ________  _________ _/ /_____  _____
  / ___/ _ \/ ___/ __ `/ __/ __ \/ ___/
 (__  /  __/ /__/ /_/ / /_/ /_/ / /    
/____/\___/\___/\__,_/\__/\____/_/     v0.6.0

                        freelabz.com
mytool -u mytarget.com -jsonl
🔗 https://mytarget.com/api [200] [application/json]
🔗 https://mytarget.com/api/metrics [403]
from secator.tasks import mytool

task = mytool('mytarget.com')
for item in task:
    print(item)  # item is now a secator output type like Vulnerability or Port
PreviousIntegrating an external commandNextParsing raw standard output

Last updated 1 month ago

Was this helpful?

Set output_types to our desired output types [].

Set output_map to map mytool's output fields to each 's fields (you need to map at least the required=True fields).

See for more details on which hooks you can use.

Lifecyle hooks
Url
Url