Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix MAC computation performance issue; add CLI entrypoint script for quickly downloading files without logging in #76

Open
wants to merge 31 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
31 commits
Select commit Hold shift + click to select a range
776e0ee
Add progress bar using tqdm
pgp Jan 18, 2022
3fc718a
Add CLI entrypoint with simple download syntax (url, and optional dow…
pgp Jan 18, 2022
bcabdad
remove pathlib from requirements.txt; raise minimum supported python …
pgp Jan 19, 2022
355191d
remove travis stuff
pgp Jan 19, 2022
26accda
replace occurrences of the old domain name mega.co.nz, with the curre…
pgp Jan 19, 2022
f292d71
fix name detection for some files
pgp Jan 24, 2022
de703c4
Update README.md
pgp Jan 24, 2022
47d30a8
remove python2 conditional definitions from crypto.py
pgp Jan 25, 2022
769cdfc
extract bytearray constant for empty iv in AES operations
pgp Jan 25, 2022
97d0f47
remove redundant .encode('utf-8') by declaring mac_str as bytes ( -> …
pgp Jan 25, 2022
2b348b8
manually throttle tqdm progress bar updates
pgp Jan 25, 2022
7d712b1
add comment
pgp Jan 25, 2022
a5a5986
add some more comments
pgp Jan 25, 2022
87bac08
add even more comments, and refactor the less-than-16-bytes-file fix
pgp Jan 25, 2022
570c6da
do not mix conditional blocks w.r.t. file size and chunk_size, just p…
pgp Jan 25, 2022
78f8d2a
MAC verification should finally work correctly now
pgp Jan 26, 2022
dd543fe
correctly close and flush buffers for temporary output file before mo…
pgp Jan 26, 2022
478886f
remove tqdm manual throttling
pgp Jan 26, 2022
15601b3
add automatic redirect to mega links from other URLs (e.g. URL shorte…
pgp Jan 28, 2022
64673c5
remove duplicated entries from .gitignore
pgp Jan 28, 2022
403a90e
make some methods static
pgp Jan 28, 2022
f5b6f5b
rename some variables in order to avoid builtin name shadowing
pgp Jan 28, 2022
f252655
remove redundant assignment
pgp Jan 28, 2022
811e3c1
remove redundant assignment related to try..except..else block as well
pgp Jan 28, 2022
acbdc01
add optional command line argument (-d,--direct) for directly downloa…
pgp Jan 28, 2022
6609e38
add comment
pgp Feb 21, 2022
00a108b
Update requirements in order to support Python 3.11
pgp Nov 17, 2022
e4abacc
fix typo; simplify if-else block
pgp Aug 6, 2023
17139e6
add a couple of comments
pgp Aug 10, 2023
d42ebe7
simplify requests.post call
pgp Aug 11, 2023
6a441d9
Raise minimum requirements (use Python 3.6 as reference)
pgp Feb 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,6 @@ local.properties
.classpath
.settings/
.loadpath
.idea
htmlcov/
# External tool builders
.externalToolBuilders/
Expand Down Expand Up @@ -64,7 +63,6 @@ htmlcov/
*.tlb
*.tli
*.tlh
*.tmp
*.vspscc
.builds
*.dotCover
Expand Down
9 changes: 0 additions & 9 deletions .travis.yml

This file was deleted.

27 changes: 9 additions & 18 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,7 @@
Mega.py
=======

[![Build
Status](https://travis-ci.org/odwyersoftware/mega.py.png?branch=master)](https://travis-ci.org/odwyersoftware/mega.py)
[![Downloads](https://pypip.in/d/mega.py/badge.png)](https://crate.io/packages/mega.py/) [![PyPI version](https://badge.fury.io/py/mega.py.svg)](https://pypi.org/project/mega.py/)

Python library for the [Mega.co.nz](https://mega.nz/aff=Zo6IxNaHw14)
Python library for the [Mega.nz](https://mega.nz)
API, currently supporting:

- login
Expand All @@ -26,14 +22,15 @@ How To Use

### Create a Mega account

First, [create an account with Mega](https://mega.nz/aff=Zo6IxNaHw14) .
For downloading links, no account is needed. To use upload capabilities, [create an account with Mega](https://mega.nz) .

### Install mega.py package

Run the following command, or run setup from the latest github source.
Clone the repository, and then run:

```python
pip install mega.py
```sh
cd mega.py
pip install .
```

### Import mega.py
Expand Down Expand Up @@ -123,7 +120,7 @@ m.upload('myfile.doc', folder[0])
```python
file = m.find('myfile.doc')
m.download(file)
m.download_url('https://mega.co.nz/#!utYjgSTQ!OM4U3V5v_W4N5edSo0wolg1D5H0fwSrLD3oLnLuS9pc')
m.download_url('https://mega.nz/#!utYjgSTQ!OM4U3V5v_W4N5edSo0wolg1D5H0fwSrLD3oLnLuS9pc')
m.download(file, '/home/john-smith/Desktop')
# specify optional download filename (download_url() supports this also)
m.download(file, '/home/john-smith/Desktop', 'myfile.zip')
Expand All @@ -132,9 +129,9 @@ m.download(file, '/home/john-smith/Desktop', 'myfile.zip')
### Import a file from URL, optionally specify destination folder

```python
m.import_public_url('https://mega.co.nz/#!utYjgSTQ!OM4U3V5v_W4N5edSo0wolg1D5H0fwSrLD3oLnLuS9pc')
m.import_public_url('https://mega.nz/#!utYjgSTQ!OM4U3V5v_W4N5edSo0wolg1D5H0fwSrLD3oLnLuS9pc')
folder_node = m.find('Documents')[1]
m.import_public_url('https://mega.co.nz/#!utYjgSTQ!OM4U3V5v_W4N5edSo0wolg1D5H0fwSrLD3oLnLuS9pc', dest_node=folder_node)
m.import_public_url('https://mega.nz/#!utYjgSTQ!OM4U3V5v_W4N5edSo0wolg1D5H0fwSrLD3oLnLuS9pc', dest_node=folder_node)
```

### Create a folder
Expand All @@ -160,9 +157,3 @@ Returns a dict of folder node name and node\_id, e.g.
file = m.find('myfile.doc')
m.rename(file, 'my_file.doc')
```

## Contact Support

For paid priority support contact [[email protected]](mailto:[email protected]).

**[UK Python Development Agency](https://odwyer.software/)**
8 changes: 4 additions & 4 deletions requirements.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
requests>=0.10
pycryptodome>=3.9.6,<4.0.0
pathlib==1.0.1
tenacity>=5.1.5,<6.0.0
requests>=2.27.1
pycryptodome>=3.20.0,<4.0.0
tenacity>=8.2.2
tqdm>=4.64.1
17 changes: 10 additions & 7 deletions setup.py
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
#!/usr/bin/env python
# -*- coding: utf-8 -*

from __future__ import absolute_import

import os
from codecs import open

Expand All @@ -22,16 +20,15 @@

setup(name='mega.py',
version='1.0.9.dev0',
python_requires='>=3.6',
packages=find_packages('src', exclude=('tests', )),
package_dir={'': 'src'},
include_package_data=True,
zip_safe=False,
url='https://github.com/odwyersoftware/mega.py',
description='Python lib for the Mega.co.nz API',
url='https://github.com/pgp/mega.py',
description='Python lib for the Mega.nz API',
long_description=readme + '\n\n' + history,
long_description_content_type='text/markdown',
author='O\'Dwyer Software',
author_email='[email protected]',
license='Creative Commons Attribution-Noncommercial-Share Alike license',
install_requires=install_requires,
classifiers=[
Expand All @@ -40,4 +37,10 @@
'Programming Language :: Python',
'Programming Language :: Python :: 3',
'Topic :: Internet :: WWW/HTTP',
])
],
entry_points={
'console_scripts': [
'meganz=mega.maincli:maincli'
]
},
)
52 changes: 29 additions & 23 deletions src/mega/crypto.py
Original file line number Diff line number Diff line change
@@ -1,37 +1,28 @@
from Crypto.Cipher import AES
import json
import base64
import struct
import binascii
import codecs
import json
import random
import sys
import struct

# Python3 compatibility
if sys.version_info < (3, ):
from Crypto.Cipher import AES

def makebyte(x):
return x
EMPTY_IV = b'\0'*16

def makestring(x):
return x
else:
import codecs
def makebyte(x):
return codecs.latin_1_encode(x)[0]

def makebyte(x):
return codecs.latin_1_encode(x)[0]

def makestring(x):
return codecs.latin_1_decode(x)[0]
def makestring(x):
return codecs.latin_1_decode(x)[0]


def aes_cbc_encrypt(data, key):
aes_cipher = AES.new(key, AES.MODE_CBC, makebyte('\0' * 16))
return aes_cipher.encrypt(data)
return AES.new(key, AES.MODE_CBC, EMPTY_IV).encrypt(data)


def aes_cbc_decrypt(data, key):
aes_cipher = AES.new(key, AES.MODE_CBC, makebyte('\0' * 16))
return aes_cipher.decrypt(data)
return AES.new(key, AES.MODE_CBC, EMPTY_IV).decrypt(data)


def aes_cbc_encrypt_a32(data, key):
Expand All @@ -42,8 +33,8 @@ def aes_cbc_decrypt_a32(data, key):
return str_to_a32(aes_cbc_decrypt(a32_to_str(data), a32_to_str(key)))


def stringhash(str, aeskey):
s32 = str_to_a32(str)
def stringhash(s, aeskey):
s32 = str_to_a32(s)
h32 = [0, 0, 0, 0]
for i in range(len(s32)):
h32[i % 4] ^= s32[i]
Expand All @@ -65,6 +56,8 @@ def prepare_key(arr):


def encrypt_key(a, key):
# this sum, which is applied to a generator of tuples, actually flattens the output list of lists of that generator
# i.e. it's equivalent to tuple([item for t in generatorOfLists for item in t])
return sum((aes_cbc_encrypt_a32(a[i:i + 4], key)
for i in range(0, len(a), 4)), ())

Expand All @@ -85,7 +78,18 @@ def decrypt_attr(attr, key):
attr = aes_cbc_decrypt(attr, a32_to_str(key))
attr = makestring(attr)
attr = attr.rstrip('\0')
return json.loads(attr[4:]) if attr[:6] == 'MEGA{"' else False

prefix = 'MEGA{"'
if attr.startswith(prefix):
i1 = 4
i2 = attr.find('}')
if i2 >= 0:
i2+=1
return json.loads(attr[i1:i2])
else:
raise RuntimeError(f'Unable to properly decode filename, raw content is: {attr}')
else:
return False


def a32_to_str(a):
Expand Down Expand Up @@ -149,6 +153,8 @@ def a32_to_base64(a):
return base64_url_encode(a32_to_str(a))


# generates a list of chunks of the kind (offset, chunk_size), where offset refers to the file start
# chunk_size starts at 0x20000 (100 KiB), and then increments linearly till saturation to 0x100000 (1 MiB)
def get_chunks(size):
p = 0
s = 0x20000
Expand Down
20 changes: 20 additions & 0 deletions src/mega/maincli.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import argparse
import os

from mega import Mega

def maincli():
parser = argparse.ArgumentParser()
parser.add_argument('-d', '--direct',
action='store_true',
dest='noTempFile',
help='Do not write to a temp file (write directly to output file)')
parser.add_argument('downloadUrl', nargs=1)
parser.add_argument('destinationDirectory', nargs='?', default=os.path.realpath('.'))
args = parser.parse_args()
m = Mega().login()
m.download_url(url=args.downloadUrl[0], dest_path=args.destinationDirectory, no_temp_file=args.noTempFile)


if __name__ == '__main__':
maincli()
Loading