Maquerade - high-performance, real-time, multi-location data obfuscation tool¶
In a nutshell Masquerade can hide your sensible data. We use it to upload our data to Artificial Intelligence clouds safely.
Why Masquerade¶
Work with data are a real need, but we can export information with sensitive data. Masquerade help you to do that.
Contributions are welcome, see CONTRIBUTING.md or skim existing tickets to see where you could help out.
Documentation¶
You can find the complete documentation at: Documentation
License¶
API-Check is Open Source and available under the Apache 2 license.
Contributions¶
Contributions are very welcome. See CONTRIBUTING.md or skim existing tickets to see where you could help out.
Contents¶
Overview¶
Masquerade has the power to read from many different locations as source, and export the obfuscated data to a different location in streaming and with a very high performance.
Examples:
- You can read data from a S3 Bucket, obfuscate data, and export results to local file system.
- You can read data from a S3 Bucket, obfuscate data, and export results to HDFS File System.
- You can read data from a HDFS Filesystem, obfuscate data, and export results to Google Cloud Storage.
- You can read data from a S3 bucket, obfuscate data, and export results to another S3 Bucket.
Overview¶
Masquerade allows different formats as input / output. Here you can find the complete list.
From MsgPack -> CSV¶
> cat binary.out | maskcsvout
Will return “hello”,”World”.
A complete usage may be:
> echo hello,World | maskcsvin | maskcsvout
Will return “hello”,”World”. Notice that our process add quotes, thats because our binary don’t know how looks the original csv, so try to build the most “correct” one.
Custom separator¶
You can use another separator like ‘|’ or ‘@’. Just provide separator param.
> echo hello@World | maskcsvin -separator '@' | maskcsvout -separator '|'
Will return “hello”|”World”.
Overview¶
Masquerade allows different locations as sources / output. Here you can find the complete list.
RabbitMQ¶
To read from Rabbit use:
> maskrabbitin -dial amqp://guest:guest@localhost:5672/ -channel test
This command will consume the queue and output the content into Standard Output.
To write on Rabbit use:
> cat data | maskrabbitout -dial amqp://guest:guest@localhost:5672/ -channel test
This command will send lines from data file into RabbitMQ.
You can copy a queue using this commands together:
> maskrabbitin -dial amqp://guest:guest@localhost:5672/ -channel topicA | maskrabbitout -dial amqp://guest:guest@localhost:5672/ -channel topicB
HDFS¶
HDFS has stdio support, just use is as follows.
To read:
> hdfs dfs -cat data.csv
To write:
> cat data.csv | hdfs dfs -put - data.csv