Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unable to Download Stories #24

Open
quietsecret123 opened this issue Oct 2, 2024 · 11 comments
Open

Unable to Download Stories #24

quietsecret123 opened this issue Oct 2, 2024 · 11 comments

Comments

@quietsecret123
Copy link

quietsecret123 commented Oct 2, 2024

  • Snapchat Downloader version: 1.4.3
  • Python version: 2.7.16
  • Operating System: Mac OS 10.15.7 (Catalina)

Description

I was trying to download stories as usual from my list but the command I normally use no longer seems to work. It was working as of 14 hours ago. Normally, when I type the command, it just starts saving stories from my list. I've tried this a few times and no luck. (Not really good at coding/tech stuff, so I don't really understand the error message... please also let me know if there are missing elements to this report and how I can find these things if needed). Thank you!

What I Did

Paste the command(s) you ran and the output.
If there was a crash, please include the traceback here.

snapchat-dl -u -i Usernames0.bat -P SnapDownloads
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 63, in _web_fetch_story
response_json = json.loads(response_json_raw[0])
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.10/bin/snapchat-dl", line 8, in
sys.exit(main())
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 58, in main
download_users(usernames)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 53, in download_users
downlaoder.download(username)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 93, in download
stories, snap_user = self._web_fetch_story(username)
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 82, in _web_fetch_story
raise APIResponseError
snapchat_dl.utils.APIResponseError

@cheese529
Copy link

I'm also getting this same error, if anyone has figured out a fix or wants to fork this project to fix this issue reach out to me please.

@promers
Copy link

promers commented Oct 2, 2024

I used chatgpt to fix it, just replace the snapchat_dl.py with this:

"""The Main Snapchat Downloader Class."""

import concurrent.futures
import json
import os
import re
import requests
from loguru import logger

from snapchat_dl.downloader import download_url
from snapchat_dl.utils import APIResponseError
from snapchat_dl.utils import dump_response
from snapchat_dl.utils import MEDIA_TYPE
from snapchat_dl.utils import NoStoriesFound
from snapchat_dl.utils import strf_time
from snapchat_dl.utils import UserNotFoundError


class SnapchatDL:
    """Interact with Snapchat API to download story."""

    def __init__(
        self,
        directory_prefix=".",
        max_workers=1,
        limit_story=-1,
        sleep_interval=1,
        quiet=False,
        dump_json=False,
    ):
        self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
        self.max_workers = max_workers
        self.limit_story = limit_story
        self.sleep_interval = sleep_interval
        self.quiet = quiet
        self.dump_json = dump_json
        self.endpoint_web = "https://www.snapchat.com/add/{}/"
        self.regexp_web_json = (
            r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
        )
        self.response_ok = requests.codes.get("ok")

    def _api_response(self, username):
        web_url = self.endpoint_web.format(username)
        headers = {
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/85.0.4183.121 Safari/537.36"
        }
        response = requests.get(web_url, headers=headers)
        return response.text

    def _web_fetch_story(self, username):
        response = self._api_response(username)
        response_json_raw = re.findall(self.regexp_web_json, response)

        if not response_json_raw:
            logger.error(f"No JSON response found for {username}. Response: {response}")
            raise APIResponseError

        try:
            response_json = json.loads(response_json_raw[0])

            def util_web_user_info(content: dict):
                if "userProfile" in content["props"]["pageProps"]:
                    user_profile = content["props"]["pageProps"]["userProfile"]
                    field_id = user_profile["$case"]
                    return user_profile[field_id]
                else:
                    raise UserNotFoundError

            def util_web_story(content: dict):
                if "story" in content["props"]["pageProps"]:
                    return content["props"]["pageProps"]["story"]["snapList"]
                return list()

            user_info = util_web_user_info(response_json)
            stories = util_web_story(response_json)
            return stories, user_info

        except (IndexError, KeyError, ValueError) as e:
            logger.error(f"Failed to fetch story for {username}. Error: {e}, Response: {response}")
            raise APIResponseError

    def download(self, username):
        """Download Snapchat Story for `username`.

        Args:
            username (str): Snapchat `username`

        Returns:
            [bool]: story downloader
        """
        stories, snap_user, *_ = self._web_fetch_story(username)

        if len(stories) == 0:
            if self.quiet is False:
                logger.info("\033[91m{}\033[0m has no stories".format(username))

            raise NoStoriesFound

        if self.limit_story > -1:
            stories = stories[0 : self.limit_story]

        logger.info("[+] {} has {} stories".format(username, len(stories)))

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
        try:
            for media in stories:
                snap_id = media["snapId"]["value"]
                media_url = media["snapUrls"]["mediaUrl"]
                media_type = media["snapMediaType"]
                timestamp = int(media["timestampInSec"]["value"])
                date_str = strf_time(timestamp, "%Y-%m-%d")

                dir_name = os.path.join(self.directory_prefix, username, date_str)

                filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
                    snap_id, username, MEDIA_TYPE[media_type]
                )

                if self.dump_json:
                    filename_json = os.path.join(dir_name, filename + ".json")
                    media_json = dict(media)
                    media_json["snapUser"] = snap_user
                    dump_response(media_json, filename_json)

                media_output = os.path.join(dir_name, filename)
                executor.submit(
                    download_url, media_url, media_output, self.sleep_interval
                )

        except KeyboardInterrupt:
            executor.shutdown(wait=False)

        logger.info("[✔] {} stories downloaded".format(username))

@quietsecret123
Copy link
Author

quietsecret123 commented Oct 3, 2024

Thanks @promers for the suggestion. I copy pasted your code and completely replaced what was in the original snapchat_dl.py using a text editor, and then saved it, but it still did not work.

Maybe I'm doing something wrong?

This is what I get when I try:

snapchat-dl foodwithmichel -P SnapDownloads

2024-10-02 20:21:03.367 | ERROR    | snapchat_dl.snapchat_dl:_web_fetch_story:73 - Failed to fetch story for foodwithmichel. Response: <!doctype html><meta charset="utf-8"><meta name=viewport content="width=device-width, initial-scale=1"><title>403</title>403 Forbidden
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 46, in _web_fetch_story
    response_json = json.loads(response_json_raw[0])
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/snapchat-dl", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 58, in main
    download_users(usernames)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 53, in download_users
    downlaoder.download(username)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 85, in download
    stories, highlights, snap_user, *_ = self._web_fetch_story(username)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 74, in _web_fetch_story
    raise APIResponseError
snapchat_dl.utils.APIResponseError

@quietsecret123
Copy link
Author

quietsecret123 commented Oct 3, 2024

Ok, thanks to @promers, I also tried Chatgpt to fix things and this is what mine came up with (and it works). Like @promers suggested, replace snapchat_dl.py with the following:

"""The Main Snapchat Downloader Class."""
import concurrent.futures
import json
import os
import re

import requests
from loguru import logger

from snapchat_dl.downloader import download_url
from snapchat_dl.utils import (
    APIResponseError,
    dump_response,
    MEDIA_TYPE,
    NoStoriesFound,
    strf_time,
    UserNotFoundError,
)


class SnapchatDL:
    """Interact with Snapchat API to download story."""

    def __init__(
        self,
        directory_prefix=".",
        max_workers=2,
        limit_story=-1,
        sleep_interval=1,
        quiet=False,
        dump_json=False,
    ):
        self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
        self.max_workers = max_workers
        self.limit_story = limit_story
        self.sleep_interval = sleep_interval
        self.quiet = quiet
        self.dump_json = dump_json
        self.endpoint_web = "https://story.snapchat.com/@{}"
        self.regexp_web_json = (
            r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
        )

    def _api_response(self, username):
        web_url = self.endpoint_web.format(username)
        headers = {
            "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.3"
        }
        return requests.get(web_url, headers=headers)

    def _web_fetch_story(self, username):
        """Download user stories from Web.

        Args:
            username (str): Snapchat `username`

        Raises:
            APIResponseError: API Error

        Returns:
            (list, dict): stories, user_info
        """
        response = self._api_response(username)

        if response.status_code != requests.codes.ok:
            logger.error(f"Failed to fetch data for {username}. Status code: {response.status_code}. "
                         f"Response: {response.text}")
            raise APIResponseError

        response_json_raw = re.findall(self.regexp_web_json, response.text)

        try:
            response_json = json.loads(response_json_raw[0])

            def util_web_user_info(content: dict):
                if "userProfile" in content["props"]["pageProps"]:
                    user_profile = content["props"]["pageProps"]["userProfile"]
                    field_id = user_profile["$case"]
                    return user_profile[field_id]
                else:
                    raise UserNotFoundError

            def util_web_story(content: dict):
                if "story" in content["props"]["pageProps"]:
                    return content["props"]["pageProps"]["story"]["snapList"]
                return list()

            user_info = util_web_user_info(response_json)
            stories = util_web_story(response_json)
            return stories, user_info
        except (IndexError, KeyError, ValueError) as e:
            logger.error(f"Error parsing response for {username}: {e}")
            raise APIResponseError

    def _download_media(self, media, username, snap_user):
        snap_id = media["snapId"]["value"]
        media_url = media["snapUrls"]["mediaUrl"]
        media_type = media["snapMediaType"]
        timestamp = int(media["timestampInSec"]["value"])
        date_str = strf_time(timestamp, "%Y-%m-%d")

        dir_name = os.path.join(self.directory_prefix, username, date_str)
        os.makedirs(dir_name, exist_ok=True)  # Ensure the directory exists

        filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
            snap_id, username, MEDIA_TYPE[media_type]
        )

        if self.dump_json:
            filename_json = os.path.join(dir_name, f"{filename}.json")
            media_json = dict(media)
            media_json["snapUser"] = snap_user
            dump_response(media_json, filename_json)

        media_output = os.path.join(dir_name, filename)
        return media_url, media_output

    def download(self, username):
        """Download Snapchat Story for `username`.

        Args:
            username (str): Snapchat `username`

        Raises:
            NoStoriesFound: If no stories are found
        """
        stories, snap_user = self._web_fetch_story(username)

        if not stories:
            if not self.quiet:
                logger.info(f"\033[91m{username}\033[0m has no stories.")
            raise NoStoriesFound

        if self.limit_story > -1:
            stories = stories[:self.limit_story]

        logger.info(f"[+] {username} has {len(stories)} stories.")

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
        try:
            for media in stories:
                media_url, media_output = self._download_media(media, username, snap_user)
                executor.submit(download_url, media_url, media_output, self.sleep_interval)

        except KeyboardInterrupt:
            executor.shutdown(wait=False)

        logger.info(f"[✔] {len(stories)} stories downloaded for {username}.")

@ne0lith
Copy link

ne0lith commented Oct 3, 2024

Alot of verbose output with 404'd users with those fixes.

Here's a fix for that too.

"""The Main Snapchat Downloader Class."""
import concurrent.futures
import json
import os
import re

import requests
from loguru import logger

from snapchat_dl.downloader import download_url
from snapchat_dl.utils import (
    APIResponseError,
    dump_response,
    MEDIA_TYPE,
    NoStoriesFound,
    strf_time,
    UserNotFoundError,
)


class SnapchatDL:
    """Interact with Snapchat API to download story."""

    def __init__(
        self,
        directory_prefix=".",
        max_workers=2,
        limit_story=-1,
        sleep_interval=1,
        quiet=False,
        dump_json=False,
    ):
        self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
        self.max_workers = max_workers
        self.limit_story = limit_story
        self.sleep_interval = sleep_interval
        self.quiet = quiet
        self.dump_json = dump_json
        self.endpoint_web = "https://story.snapchat.com/@{}"
        self.regexp_web_json = (
            r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
        )

    def _api_response(self, username):
        web_url = self.endpoint_web.format(username)
        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/58.0.3029.110 Safari/537.3"
            )
        }
        return requests.get(web_url, headers=headers)

    def _web_fetch_story(self, username):
        """Download user stories from Web.

        Args:
            username (str): Snapchat `username`

        Raises:
            APIResponseError: API Error

        Returns:
            (list, dict): stories, user_info
        """
        response = self._api_response(username)

        if response.status_code != requests.codes.ok:
            if response.status_code == 404:
                logger.error(f"User '{username}' not found (404).")
            else:
                logger.error(f"Failed to fetch data for '{username}'. Status code: {response.status_code}.")
            raise APIResponseError

        response_json_raw = re.findall(self.regexp_web_json, response.text)

        try:
            response_json = json.loads(response_json_raw[0])

            def util_web_user_info(content: dict):
                if "userProfile" in content["props"]["pageProps"]:
                    user_profile = content["props"]["pageProps"]["userProfile"]
                    field_id = user_profile["$case"]
                    return user_profile[field_id]
                else:
                    raise UserNotFoundError

            def util_web_story(content: dict):
                if "story" in content["props"]["pageProps"]:
                    return content["props"]["pageProps"]["story"]["snapList"]
                return list()

            user_info = util_web_user_info(response_json)
            stories = util_web_story(response_json)
            return stories, user_info
        except (IndexError, KeyError, ValueError) as e:
            logger.error(f"Error parsing response for '{username}': {e}")
            raise APIResponseError

    def _download_media(self, media, username, snap_user):
        snap_id = media["snapId"]["value"]
        media_url = media["snapUrls"]["mediaUrl"]
        media_type = media["snapMediaType"]
        timestamp = int(media["timestampInSec"]["value"])
        date_str = strf_time(timestamp, "%Y-%m-%d")

        dir_name = os.path.join(self.directory_prefix, username, date_str)
        os.makedirs(dir_name, exist_ok=True)  # Ensure the directory exists

        filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
            snap_id, username, MEDIA_TYPE[media_type]
        )

        if self.dump_json:
            filename_json = os.path.join(dir_name, f"{filename}.json")
            media_json = dict(media)
            media_json["snapUser"] = snap_user
            dump_response(media_json, filename_json)

        media_output = os.path.join(dir_name, filename)
        return media_url, media_output

    def download(self, username):
        """Download Snapchat Story for `username`.

        Args:
            username (str): Snapchat `username`
        """
        try:
            stories, snap_user = self._web_fetch_story(username)
        except APIResponseError:
            logger.error(f"Could not fetch data for '{username}'. The user may not exist or has no public stories.")
            return

        if not stories:
            if not self.quiet:
                logger.info(f"{username} has no stories.")
            return

        logger.info(f"[+] {username} has {len(stories)} stories.")

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
        try:
            for media in stories:
                media_url, media_output = self._download_media(media, username, snap_user)
                executor.submit(download_url, media_url, media_output, self.sleep_interval)
        except KeyboardInterrupt:
            executor.shutdown(wait=False)

        logger.info(f"[✔] {len(stories)} stories downloaded for {username}.")

@quietsecret123
Copy link
Author

quietsecret123 commented Oct 3, 2024

Thanks for the correction, @ne0lith. It works better now!

Interestingly, some SnapStar profiles (have golden stars beside their usernames) have stories when viewed on Snap but not when their profile is viewed on the web (i.e. as https://www.snapchat.com/add/username). As such, snapchat-dl is unable to download their stories. Is there any workaround for those?

@promers
Copy link

promers commented Oct 3, 2024

Thanks @promers for the suggestion. I copy pasted your code and completely replaced what was in the original snapchat_dl.py using a text editor, and then saved it, but it still did not work.

Maybe I'm doing something wrong?

This is what I get when I try:

snapchat-dl foodwithmichel -P SnapDownloads

2024-10-02 20:21:03.367 | ERROR    | snapchat_dl.snapchat_dl:_web_fetch_story:73 - Failed to fetch story for foodwithmichel. Response: <!doctype html><meta charset="utf-8"><meta name=viewport content="width=device-width, initial-scale=1"><title>403</title>403 Forbidden
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 46, in _web_fetch_story
    response_json = json.loads(response_json_raw[0])
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.10/bin/snapchat-dl", line 8, in <module>
    sys.exit(main())
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 58, in main
    download_users(usernames)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/app.py", line 53, in download_users
    downlaoder.download(username)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 85, in download
    stories, highlights, snap_user, *_ = self._web_fetch_story(username)
  File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/snapchat_dl/snapchat_dl.py", line 74, in _web_fetch_story
    raise APIResponseError
snapchat_dl.utils.APIResponseError

Oops my bad. Even though it worked on my end, it's seems that chatgpt did not fully update the code when i tried to add highlights. Glad you fixed it.

@Dreamscape-419
Copy link

Alot of verbose output with 404'd users with those fixes.

Here's a fix for that too.

"""The Main Snapchat Downloader Class."""
import concurrent.futures
import json
import os
import re

import requests
from loguru import logger

from snapchat_dl.downloader import download_url
from snapchat_dl.utils import (
    APIResponseError,
    dump_response,
    MEDIA_TYPE,
    NoStoriesFound,
    strf_time,
    UserNotFoundError,
)


class SnapchatDL:
    """Interact with Snapchat API to download story."""

    def __init__(
        self,
        directory_prefix=".",
        max_workers=2,
        limit_story=-1,
        sleep_interval=1,
        quiet=False,
        dump_json=False,
    ):
        self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
        self.max_workers = max_workers
        self.limit_story = limit_story
        self.sleep_interval = sleep_interval
        self.quiet = quiet
        self.dump_json = dump_json
        self.endpoint_web = "https://story.snapchat.com/@{}"
        self.regexp_web_json = (
            r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
        )

    def _api_response(self, username):
        web_url = self.endpoint_web.format(username)
        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/58.0.3029.110 Safari/537.3"
            )
        }
        return requests.get(web_url, headers=headers)

    def _web_fetch_story(self, username):
        """Download user stories from Web.

        Args:
            username (str): Snapchat `username`

        Raises:
            APIResponseError: API Error

        Returns:
            (list, dict): stories, user_info
        """
        response = self._api_response(username)

        if response.status_code != requests.codes.ok:
            if response.status_code == 404:
                logger.error(f"User '{username}' not found (404).")
            else:
                logger.error(f"Failed to fetch data for '{username}'. Status code: {response.status_code}.")
            raise APIResponseError

        response_json_raw = re.findall(self.regexp_web_json, response.text)

        try:
            response_json = json.loads(response_json_raw[0])

            def util_web_user_info(content: dict):
                if "userProfile" in content["props"]["pageProps"]:
                    user_profile = content["props"]["pageProps"]["userProfile"]
                    field_id = user_profile["$case"]
                    return user_profile[field_id]
                else:
                    raise UserNotFoundError

            def util_web_story(content: dict):
                if "story" in content["props"]["pageProps"]:
                    return content["props"]["pageProps"]["story"]["snapList"]
                return list()

            user_info = util_web_user_info(response_json)
            stories = util_web_story(response_json)
            return stories, user_info
        except (IndexError, KeyError, ValueError) as e:
            logger.error(f"Error parsing response for '{username}': {e}")
            raise APIResponseError

    def _download_media(self, media, username, snap_user):
        snap_id = media["snapId"]["value"]
        media_url = media["snapUrls"]["mediaUrl"]
        media_type = media["snapMediaType"]
        timestamp = int(media["timestampInSec"]["value"])
        date_str = strf_time(timestamp, "%Y-%m-%d")

        dir_name = os.path.join(self.directory_prefix, username, date_str)
        os.makedirs(dir_name, exist_ok=True)  # Ensure the directory exists

        filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
            snap_id, username, MEDIA_TYPE[media_type]
        )

        if self.dump_json:
            filename_json = os.path.join(dir_name, f"{filename}.json")
            media_json = dict(media)
            media_json["snapUser"] = snap_user
            dump_response(media_json, filename_json)

        media_output = os.path.join(dir_name, filename)
        return media_url, media_output

    def download(self, username):
        """Download Snapchat Story for `username`.

        Args:
            username (str): Snapchat `username`
        """
        try:
            stories, snap_user = self._web_fetch_story(username)
        except APIResponseError:
            logger.error(f"Could not fetch data for '{username}'. The user may not exist or has no public stories.")
            return

        if not stories:
            if not self.quiet:
                logger.info(f"{username} has no stories.")
            return

        logger.info(f"[+] {username} has {len(stories)} stories.")

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
        try:
            for media in stories:
                media_url, media_output = self._download_media(media, username, snap_user)
                executor.submit(download_url, media_url, media_output, self.sleep_interval)
        except KeyboardInterrupt:
            executor.shutdown(wait=False)

        logger.info(f"[✔] {len(stories)} stories downloaded for {username}.")

Thanks for this. I was also having the same issue and this resolved it.

@cheese529
Copy link

Alot of verbose output with 404'd users with those fixes.

Here's a fix for that too.

"""The Main Snapchat Downloader Class."""
import concurrent.futures
import json
import os
import re

import requests
from loguru import logger

from snapchat_dl.downloader import download_url
from snapchat_dl.utils import (
    APIResponseError,
    dump_response,
    MEDIA_TYPE,
    NoStoriesFound,
    strf_time,
    UserNotFoundError,
)


class SnapchatDL:
    """Interact with Snapchat API to download story."""

    def __init__(
        self,
        directory_prefix=".",
        max_workers=2,
        limit_story=-1,
        sleep_interval=1,
        quiet=False,
        dump_json=False,
    ):
        self.directory_prefix = os.path.abspath(os.path.normpath(directory_prefix))
        self.max_workers = max_workers
        self.limit_story = limit_story
        self.sleep_interval = sleep_interval
        self.quiet = quiet
        self.dump_json = dump_json
        self.endpoint_web = "https://story.snapchat.com/@{}"
        self.regexp_web_json = (
            r'<script\s*id="__NEXT_DATA__"\s*type="application\/json">([^<]+)<\/script>'
        )

    def _api_response(self, username):
        web_url = self.endpoint_web.format(username)
        headers = {
            "User-Agent": (
                "Mozilla/5.0 (Windows NT 10.0; Win64; x64) "
                "AppleWebKit/537.36 (KHTML, like Gecko) "
                "Chrome/58.0.3029.110 Safari/537.3"
            )
        }
        return requests.get(web_url, headers=headers)

    def _web_fetch_story(self, username):
        """Download user stories from Web.

        Args:
            username (str): Snapchat `username`

        Raises:
            APIResponseError: API Error

        Returns:
            (list, dict): stories, user_info
        """
        response = self._api_response(username)

        if response.status_code != requests.codes.ok:
            if response.status_code == 404:
                logger.error(f"User '{username}' not found (404).")
            else:
                logger.error(f"Failed to fetch data for '{username}'. Status code: {response.status_code}.")
            raise APIResponseError

        response_json_raw = re.findall(self.regexp_web_json, response.text)

        try:
            response_json = json.loads(response_json_raw[0])

            def util_web_user_info(content: dict):
                if "userProfile" in content["props"]["pageProps"]:
                    user_profile = content["props"]["pageProps"]["userProfile"]
                    field_id = user_profile["$case"]
                    return user_profile[field_id]
                else:
                    raise UserNotFoundError

            def util_web_story(content: dict):
                if "story" in content["props"]["pageProps"]:
                    return content["props"]["pageProps"]["story"]["snapList"]
                return list()

            user_info = util_web_user_info(response_json)
            stories = util_web_story(response_json)
            return stories, user_info
        except (IndexError, KeyError, ValueError) as e:
            logger.error(f"Error parsing response for '{username}': {e}")
            raise APIResponseError

    def _download_media(self, media, username, snap_user):
        snap_id = media["snapId"]["value"]
        media_url = media["snapUrls"]["mediaUrl"]
        media_type = media["snapMediaType"]
        timestamp = int(media["timestampInSec"]["value"])
        date_str = strf_time(timestamp, "%Y-%m-%d")

        dir_name = os.path.join(self.directory_prefix, username, date_str)
        os.makedirs(dir_name, exist_ok=True)  # Ensure the directory exists

        filename = strf_time(timestamp, "%Y-%m-%d_%H-%M-%S {} {}.{}").format(
            snap_id, username, MEDIA_TYPE[media_type]
        )

        if self.dump_json:
            filename_json = os.path.join(dir_name, f"{filename}.json")
            media_json = dict(media)
            media_json["snapUser"] = snap_user
            dump_response(media_json, filename_json)

        media_output = os.path.join(dir_name, filename)
        return media_url, media_output

    def download(self, username):
        """Download Snapchat Story for `username`.

        Args:
            username (str): Snapchat `username`
        """
        try:
            stories, snap_user = self._web_fetch_story(username)
        except APIResponseError:
            logger.error(f"Could not fetch data for '{username}'. The user may not exist or has no public stories.")
            return

        if not stories:
            if not self.quiet:
                logger.info(f"{username} has no stories.")
            return

        logger.info(f"[+] {username} has {len(stories)} stories.")

        executor = concurrent.futures.ThreadPoolExecutor(max_workers=self.max_workers)
        try:
            for media in stories:
                media_url, media_output = self._download_media(media, username, snap_user)
                executor.submit(download_url, media_url, media_output, self.sleep_interval)
        except KeyboardInterrupt:
            executor.shutdown(wait=False)

        logger.info(f"[✔] {len(stories)} stories downloaded for {username}.")

ok so i downloaded the source code and then i edited the snapchat_dl.py file and copy pasted this code. how do i actually run the program now? (which directory do i have to cmd into and what command do i use?)

@ne0lith
Copy link

ne0lith commented Oct 4, 2024

You dont need to download the source code manually.
Install it via pip like normal, then locate your copy of snapchat_dl.py that pip gave you.

Something like this:
"C:\Users\USERNAME\AppData\Local\Programs\Python\Python311\Lib\site-packages\snapchat_dl\snapchat_dl.py"

Basically the editable file will be in your site-packages/snapchat_dl directory.

JaKrIvMa added a commit to JaKrIvMa/snapchat-dlp that referenced this issue Oct 4, 2024
Fixing download thanks to

skyme5#24 (comment)
@Tonylocoz
Copy link

Thanks for this guys, can someone let me know if you can download stories like these? https://www.snapchat.com/p/6f098158-691e-46b4-a82d-6a98b3f15dc8/3298905170300928

When I input the username it says it has no stories so I am not exactly sure what is going on.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

6 participants