Parsing JSON lines
... or how to integrate tools that output JSON lines.
If your tool outputs JSON lines, it's very easy to integrate it with secator
.
Based on how your tool's output maps will map to secator
output types, read:
One-to-one mapping (1 JSON line = 1
secator
output type).One-to-many (1 JSON line = many
secator
output types).
One-to-one mapping
Steps:
Use the
JSONSerializer
item loader.Add an
output_map
to map your tool's output to one of secator's Output types.
Example:
For instance, mytool
outputs JSON lines when we run it like:
mytool -jsonl -u mytarget.com
{"url": "https://mytarget.com/api", "status": 200, {"details": {"ct": "application/json"}}
{"url": "https://mytarget.com/api/metrics", "status": 403, "details": {}}
An integration of mytool
with secator
would look like:
from secator.decorators import task
from secator.runners import Command
from secator.output_types import Url
from secator.serializers import JSONSerializer
from secator.definitions import URL, STATUS_CODE, CONTENT_TYPE
@task()
class mytool(Command):
input_flag = '-u'
json_flag = '-jsonl'
output_types = [Url]
item_loaders = [JSONSerializer()]
output_map = {
Url: {
URL: 'url',
STATUS_CODE: 'status',
CONTENT_TYPE: lambda x: x['details'].get('ct', '')
}
}
Steps taken:
Import the desired runner (
Command
).Import the desired output type(s) and the field names that we want to map.
Set
input_flag
tomytool
's input flag-u
.Set
json_flag
tomytool
's JSON line flagjsonl
.Set
output_types
to our desired output types [Url
].Set
output_map
to mapmytool
's output fields to eachUrl
's fields (you need to map at least therequired=True
fields).
That's it ! You can now run mytool
with secator
and enjoy all the features it brings on top of it:
secator x mytool mytarget.com
__
________ _________ _/ /_____ _____
/ ___/ _ \/ ___/ __ `/ __/ __ \/ ___/
(__ / __/ /__/ /_/ / /_/ /_/ / /
/____/\___/\___/\__,_/\__/\____/_/ v0.6.0
freelabz.com
mytool -u mytarget.com -jsonl
🔗 https://mytarget.com/api [200] [application/json]
🔗 https://mytarget.com/api/metrics [403]
One-to-many
Steps:
Add a static method
on_json_loaded
to hook onto the JSON Serializer output.Modify the data and yield
secator
Output types.
Example:
For instance, if mytool
outputs urls in batch:
mytool -jsonl -u mytarget.com
{"urls": [{"url": "https://mytarget.com/api", "status": 200, "content-type": "application/json"}, {"url": "https://mytarget.com/api/metrics", "status": 403, "content-type": "application/text"}]
An integration of mytool
with secator
would look like:
from secator.decorators import task
from secator.runners import Command
from secator.output_types import Url
from secator.definitions import URL, STATUS_CODE, CONTENT_TYPE
@task()
class mytool(Command):
input_flag = '-u'
json_flag = '-jsonl'
output_types = [Url]
@staticmethod
def on_json_loaded(self, data):
for item in data['urls']:
yield Url(
URL: item['url'],
STATUS_CODE: item['status'],
CONTENT_TYPE: item.get('content-type', '')
)
That's it ! You can now run mytool
with secator
and enjoy all the features it brings on top of it:
secator x mytool mytarget.com
__
________ _________ _/ /_____ _____
/ ___/ _ \/ ___/ __ `/ __/ __ \/ ___/
(__ / __/ /__/ /_/ / /_/ /_/ / /
/____/\___/\___/\__,_/\__/\____/_/ v0.6.0
freelabz.com
mytool -u mytarget.com -jsonl
🔗 https://mytarget.com/api [200] [application/json]
🔗 https://mytarget.com/api/metrics [403]
Last updated
Was this helpful?