Skip to content

DANancy/Animal-Crossing-API-Scraper

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

28 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Animal-Crossing-API-Scraper

This project is built to extract jsons and images for fish & bugs from animal crossing API, and upload to Cloud Service or send emails automatically.

Support Service

  • Google Drive: Allow you to copy a entire folder to the Google Drive automatically
  • Amazon S3: Allow you to watch the entire folder and sync automatically to AWS S3
  • SMTP Server: Allow you to send emails with attachment to groups of recipients automatically

ACNH API Scraper Architecture

Project Overview

.
├── ACNH_API                   # Project files for ANCH API
│   ├── cloud_verification     # self-defined module
│   ├── api_extractor          # spider for ACNH API
│   ├── google_drive_uploader  # upload all files to Google Drive automatically
│   ├── amazon_s3_cli          # upload all files to Amazon S3 automatically using CLI tool 
│   ├── emails_distributor     # zip all files and distribute emails automatically
│   └── api_sample.json        # animal crossing api call sample
├── images                     # images for diagram 
├── .env-sample                # Rename to .env to setup environmental variables             
├── LICENSE
├── .gitignore
├── requirements.txt
└── README.md

Setup

1.Clone the whole project

$ git clone https://github.com/DANancy/Animal-Crossing-API-Scraper.git

2.Run API Extractor

$ python ACNH_API/api_extractor.py

3.Setup Cloud Server/SMTP Server

4.Additional Setup

For Google Drive: Add Self-Defined Module -> cloud_verification.py

import pandas
# get package folder
print(pandas.__file__)
# copy this module to the package folder

For Amazon/SMTP Server: Create .env File

# SMTP Server Setup 
MAIL_SERVER=smtp.gmail.com
MAIL_USE_TLS=True
MAIL_USE_SST=False
MAIL_PORT=587
MAIL_USERNAME=example@gmail.com
MAIL_PASSWORD=*****
RECIPIENTS=["example@gmail.com","example1@gmail.com","example2@gmail.com"]

# AWS Setup
AWS_ACCESS_KEY_ID=*****
AWS_SECRET_ACCESS_KEY=*****
AWS_DEFAULT_REGION=ap-southeast-2
LocalFolder=data
AWSBucket=data

5.Run Uploader based on your needs

Upload folder to Google Drive

$ python ACNH_API/google_drive_uploader.py

Sync data to AWS S3

$ python ACNH_API/amzon_s3_cli.py

Send emails with attachment

$ python ACNH_API/emails_distributor.py

References

Credits

  • Mutoo - Wash dishes and take care of baby while I am programming :D

About

Enjoy Animal Crossing

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages