A tool to determine the content type of a file with deep learning

Navigation

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for invernizzi from gravatar.com invernizzi Avatar for reyammer from gravatar.com reyammer

Unverified details

These details have not been verified by PyPI
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Magika Developers
  • Tags content type detection , machine learning
  • Requires: Python >=3.8
Classifiers
Report project as malware

Project description

Magika Python Package

image NPM Version image image OpenSSF Best Practices CodeQL Actions status PyPI Monthly Downloads PyPI Downloads

Magika is a novel AI-powered file type detection tool that relies on the recent advance of deep learning to provide accurate detection. Under the hood, Magika employs a custom, highly optimized model that only weighs about a few MBs, and enables precise file identification within milliseconds, even when running on a single CPU. Magika has been trained and evaluated on a dataset of ~100M samples across 200+ content types (covering both binary and textual file formats), and it achieves an average ~99% accuracy on our test set.

Use Magika as a command line client or in your Python code!

You can find more information on which content types are supported, extended documentation, and bindings for other languages on Magika's website at https://securityresearch.google/magika/.

Installing Magika

Magika is available as magika on PyPI:

To install the most recent stable version:

pip install magika

If you intend to use Magika only as a command line, you may want to use pipx install magika instead.

If you want to test out the latest release candidate, you can install it with pip install --pre magika.

Using Magika as a command-line tool

Beginning with version 0.6.0, the magika Python package includes a pre-compiled Rust-based command-line tool, replacing the previous Python version. This binary is distributed as platform-specific wheels for most common architectures. For unsupported platforms, a pure-Python wheel is also available, providing the legacy Python client as a fallback.

$ cd tests_data/basic && magika -r *
asm/code.asm: Assembly (code)
batch/simple.bat: DOS batch file (code)
c/code.c: C source (code)
css/code.css: CSS source (code)
csv/magika_test.csv: CSV document (code)
dockerfile/Dockerfile: Dockerfile (code)
docx/doc.docx: Microsoft Word 2007+ document (document)
epub/doc.epub: EPUB document (document)
epub/magika_test.epub: EPUB document (document)
flac/test.flac: FLAC audio bitstream data (audio)
handlebars/example.handlebars: Handlebars source (code)
html/doc.html: HTML document (code)
ini/doc.ini: INI configuration file (text)
javascript/code.js: JavaScript source (code)
jinja/example.j2: Jinja template (code)
jpeg/magika_test.jpg: JPEG image data (image)
json/doc.json: JSON document (code)
latex/sample.tex: LaTeX document (text)
makefile/simple.Makefile: Makefile source (code)
markdown/README.md: Markdown document (text)
[...]

$ magika ./tests_data/basic/python/code.py --json
[
  {
    "path": "./tests_data/basic/python/code.py",
    "result": {
      "status": "ok",
      "value": {
        "dl": {
          "description": "Python source",
          "extensions": [
            "py",
            "pyi"
          ],
          "group": "code",
          "is_text": true,
          "label": "python",
          "mime_type": "text/x-python"
        },
        "output": {
          "description": "Python source",
          "extensions": [
            "py",
            "pyi"
          ],
          "group": "code",
          "is_text": true,
          "label": "python",
          "mime_type": "text/x-python"
        },
        "score": 0.753000020980835
      }
    }
  }
]

$ cat doc.ini | magika -
-: INI configuration file (text)

$ magika --help
Determines the content type of files with deep-learning

Usage: magika [OPTIONS] [PATH]...

Arguments:
  [PATH]...
          List of paths to the files to analyze.

          Use a dash (-) to read from standard input (can only be used once).

Options:
  -r, --recursive
          Identifies files within directories instead of identifying the directory itself

      --no-dereference
          Identifies symbolic links as is instead of identifying their content by following them

      --colors
          Prints with colors regardless of terminal support

      --no-colors
          Prints without colors regardless of terminal support

  -s, --output-score
          Prints the prediction score in addition to the content type

  -i, --mime-type
          Prints the MIME type instead of the content type description

  -l, --label
          Prints a simple label instead of the content type description

      --json
          Prints in JSON format

      --jsonl
          Prints in JSONL format

      --format <CUSTOM>
          Prints using a custom format (use --help for details).

          The following placeholders are supported:

            %p  The file path
            %l  The unique label identifying the content type
            %d  The description of the content type
            %g  The group of the content type
            %m  The MIME type of the content type
            %e  Possible file extensions for the content type
            %s  The score of the content type for the file
            %S  The score of the content type for the file in percent
            %b  The model output if overruled (empty otherwise)
            %%  A literal %

  -h, --help
          Print help (see a summary with '-h')

  -V, --version
          Print version

Check the Rust CLI docs for more information.

Using Magika as a Python module

Here is a few examples on how to use the Magika Python module:

>>> from magika import Magika
>>> m = Magika()
>>> res = m.identify_bytes(b'function log(msg) {console.log(msg);}')
>>> print(res.output.label)
javascript

>>> from magika import Magika
>>> m = Magika()
>>> res = m.identify_path('./tests_data/basic/ini/doc.ini')
>>> print(res.output.label)
ini

>>> from magika import Magika
>>> m = Magika()
>>> with open('./tests_data/basic/ini/doc.ini', 'rb') as f:
>>>     res = m.identify_stream(f)
>>> print(res.output.label)
ini

Core Concepts

To get the most out of Magika, it's worth learning about its core concepts. You can read about how Magika works, the models, the supported content types, the prediction modes, and notes on how to understand all parts of the output in the Core Concepts section of Magika's website.

API documentation

The Python API is documented in the Python bindings section of Magika's website, and via docstrings in the Magika module source code.

Research Paper and Citation

We describe how we developed Magika and the choices we made in our research paper, which was accepted at the International Conference on Software Engineering (ICSE) 2025. You can find a copy of the paper here.

If you use this software for your research, please cite it as:

@InProceedings{fratantonio25:magika,
  author = {Yanick Fratantonio and Luca Invernizzi and Loua Farah and Kurt Thomas and Marina Zhang and Ange Albertini and Francois Galilee and Giancarlo Metitieri and Julien Cretin and Alexandre Petit-Bianco and David Tao and Elie Bursztein},
  title = {{Magika: AI-Powered Content-Type Detection}},
  booktitle = {Proceedings of the International Conference on Software Engineering (ICSE)},
  month = {April},
  year = {2025}
}

Project details

Verified details

These details have been verified by PyPI
Project links
GitHub Statistics
Maintainers
Avatar for invernizzi from gravatar.com invernizzi Avatar for reyammer from gravatar.com reyammer

Unverified details

These details have not been verified by PyPI
Meta
  • License: Apache Software License (Apache-2.0)
  • Author: Magika Developers
  • Tags content type detection , machine learning
  • Requires: Python >=3.8
Classifiers

Release history Release notifications | RSS feed

This version

1.0.1

0.6.3

0.6.3rc1 pre-release

0.6.3rc0 pre-release

0.6.2

0.6.1

0.6.1rc3 pre-release

0.6.1rc2 pre-release

0.6.1rc1 pre-release

0.6.1rc0 pre-release

0.6.0rc3 pre-release

0.6.0rc2 pre-release

0.5.1

0.5.0

0.1.0 yanked

Reason this release was yanked:

Placeholder package - please upgrade

Download files

Download the file for your platform. If you're not sure which to choose, learn more about installing packages.

Source Distribution

magika-1.0.1.tar.gz (3.0 MB view details)

Uploaded Source

Built Distributions

If you're not sure about the file name format, learn more about wheel file names.

magika-1.0.1-py3-none-win_amd64.whl (12.7 MB view details)

Uploaded Python 3Windows x86-64

magika-1.0.1-py3-none-manylinux_2_28_x86_64.whl (15.4 MB view details)

Uploaded Python 3manylinux: glibc 2.28+ x86-64

magika-1.0.1-py3-none-macosx_11_0_arm64.whl (13.4 MB view details)

Uploaded Python 3macOS 11.0+ ARM64

magika-1.0.1-py3-none-any.whl (3.0 MB view details)

Uploaded Python 3

File details

Details for the file magika-1.0.1.tar.gz.

File metadata

  • Download URL: magika-1.0.1.tar.gz
  • Upload date:
  • Size: 3.0 MB
  • Tags: Source
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for magika-1.0.1.tar.gz
Algorithm Hash digest
SHA256 313f8cbfcdc9a7e55c24286273230a2732b89825e58a93de2b576531393b8398
MD5 7ca8721e735efcdeacec3431a6675f5f
BLAKE2b-256 721ad2cedf4e7fe98d62f444b17925eb9e61664e0225ceb27f39e30b73409404

See more details on using hashes here.

Provenance

The following attestation bundles were made for magika-1.0.1.tar.gz:

Publisher: python-build-and-release-package.yml on google/magika

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file magika-1.0.1-py3-none-win_amd64.whl.

File metadata

  • Download URL: magika-1.0.1-py3-none-win_amd64.whl
  • Upload date:
  • Size: 12.7 MB
  • Tags: Python 3, Windows x86-64
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for magika-1.0.1-py3-none-win_amd64.whl
Algorithm Hash digest
SHA256 1697e89aa338ea43b389c46652d0c96485e9ec2b72e8562c7d13ec31ff946911
MD5 05a069a84560ee40c28d9e7d0c7c258b
BLAKE2b-256 afa64a71d6d1b0c9f076f811089fc693d5ddcc419695bf41869a710e84307413

See more details on using hashes here.

Provenance

The following attestation bundles were made for magika-1.0.1-py3-none-win_amd64.whl:

Publisher: python-build-and-release-package.yml on google/magika

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file magika-1.0.1-py3-none-manylinux_2_28_x86_64.whl.

File metadata

File hashes

Hashes for magika-1.0.1-py3-none-manylinux_2_28_x86_64.whl
Algorithm Hash digest
SHA256 69457018fc4e5030523b6505e3c8d58e24d0a01985c21ba1ea9387f84e9138d0
MD5 fbe477ba77f84148dc49963fdca30166
BLAKE2b-256 d798936f2393de5d2d687ee256b41c5db67e07732b12b07777b79552321befb9

See more details on using hashes here.

Provenance

The following attestation bundles were made for magika-1.0.1-py3-none-manylinux_2_28_x86_64.whl:

Publisher: python-build-and-release-package.yml on google/magika

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file magika-1.0.1-py3-none-macosx_11_0_arm64.whl.

File metadata

File hashes

Hashes for magika-1.0.1-py3-none-macosx_11_0_arm64.whl
Algorithm Hash digest
SHA256 efd29299bf577227dd6b2c895a1bb2fac5b012f48492c95fc23eb5e037f0d12b
MD5 27b0679c1aa58a9b2ff8d772b1d3a633
BLAKE2b-256 cf54b2bd0d96293afb47570eeadd6506977f70df6a1f1a08e897c07225bde717

See more details on using hashes here.

Provenance

The following attestation bundles were made for magika-1.0.1-py3-none-macosx_11_0_arm64.whl:

Publisher: python-build-and-release-package.yml on google/magika

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

File details

Details for the file magika-1.0.1-py3-none-any.whl.

File metadata

  • Download URL: magika-1.0.1-py3-none-any.whl
  • Upload date:
  • Size: 3.0 MB
  • Tags: Python 3
  • Uploaded using Trusted Publishing? Yes
  • Uploaded via: twine/6.1.0 CPython/3.13.7

File hashes

Hashes for magika-1.0.1-py3-none-any.whl
Algorithm Hash digest
SHA256 efb384e87ee779c40f2343d33ff38cded807d71a9e228e7c1fabf2a988831369
MD5 eb34392e8bb7fe5578743692c2724086
BLAKE2b-256 3dd97a1a5b8148ae56782f9b7f600c10faa7c08fae07420364fc2097176531b7

See more details on using hashes here.

Provenance

The following attestation bundles were made for magika-1.0.1-py3-none-any.whl:

Publisher: python-build-and-release-package.yml on google/magika

Attestations: Values shown here reflect the state when the release was signed and may no longer be current.

Supported by

AWS Cloud computing and Security Sponsor Datadog Monitoring Depot Continuous Integration Fastly CDN Google Download Analytics Pingdom Monitoring Sentry Error logging StatusPage Status page