... or how to integrate groups of tasks with similar options.
This section will present a more complex use case where we have three commands: bigdog, catkiller and eagle which purpose is to find cats.
We will start by integrating bigdog to secator before realizing that most options can be mutualized between the three tools, and a common output type Cat can be created for all three.
Bigdog
Let's suppose we have a fictional utility called bigdog which purpose is to hunt cats on the internet. We want to add bigdog to secator.
A basic definition of bigdog using basic secator concepts will be:
secator/tasks/bigdog.py
from secator.runners import Command
from secator.decorators import task
@task
class bigdog(Command):
cmd = 'bigdog'
json_flag = '-json'
input_flag = '-site'
file_flag = '-list'
You can now run bigdog from the CLI or the library:
secator x bigdog --help
secator x bigdog loadsofcats.com
from secator.tasks import bigdog
# Get all results as a list, blocks until command has finished running
bigdog('loadsofcats.com').run()
[
{"name": "garfield", "age": 14, "host": "loadofcats.com", "position": "boss"},
{"name": "tony", "age": 18, "host": "loadsofcats.com", "position": "admin"}
]
# Get result items in real-time as they arrive to stdout
for cat in bigdog('loadsofcats.com'):
print(cat['name'] + '(' + cat['age'] + ')')
# Will print
garfield (14)
tony (18)
Okay, this is a good start.
Now what if the bigdog command has some more options that you would like to integrate ?
-timeout allows to specify a request timeout.
-rate allows to specify the max requests per minute.
You can add the opts parameter to your Command object to define the cmd options:
from secator.runners import Command
from secator.decorators import task
@task
class bigdog(Command):
cmd = 'bigdog'
json_flag = '-json'
input_flag = '-site'
file_flag = '-list'
opt_prefix = '-'
opts = {
'timeout': {'type': int, 'help': 'Timeout (in seconds)'},
'rate': {'type': int, 'help': 'Max requests per minute'}
}
You can now use bigdog with this set of options:
secator x bigdog --help
secator x bigdog loadsofcats.com -json
secator x bigdog loadsofcats.com -timeout 1 -rate 100 -o table,csv,txt,gdrive
from secator.tasks import bigdog
bigdog('loadsofcats.com', rate=100, timeout=3).run() # adding rate and timeout options
Cat hunters category
One advantage of having class-based definitions is that we can group similar tools together in categories.
Let's assume we have 2 other tools that can hunt cats: catkiller and eagle...
... but each of those tools might be written by a different person, and so the interface and output is different for each of them:
$ catkiller --host loadsofcats.com --max-wait 1000 --max-rate 10 --json
Starting catkiller session ...
{"_info": {"name": "tony", "years": 18}, "site": "loadsofcats.com", "job": "admin"}
{"_info": {"name": "garfield", "years": 14}, "site": "loadsofcats.com", "job": "boss"}
# or to pass multiple hosts, it needs to be called like:
$ cat hosts.txt | catkiller --max-wait 1000 --max-rate 10 --json
Inputs:
--host is equivalent to bigdog's -site.
--max-wait is equivalent to bigdog's -timeout, but in milliseconds instead of seconds.
--max-rate is equivalent to bigdog's -rate.
--json is equivalent to bigdog's-json option, but uses a different option character "--".
cat hosts.txt | catkiller is the equivalent tobigdog's -list.
Output:
_info has the data for name and age, but age is now years.
