How I download +2k photos from Google to display in a photo frame Part I
By Alberto on May 17, 2020
A project that I had in mind during these days is the creation of a digital portrait, in which I can show all the images that I have saved in Google Photos. But not really all, if not use some kind of filter and automation.
For this purpose and with the aim of practicing a bit of the Go programming language and implementing my knowledge in databases and microservices. I decided to get down to work this weekend to build the core of this service. Which has as functions:
• Connect with Google API and download all the metadata of the photos for a specific album.
• Enter the metadata in the database (I have used MongoDB).
• Download images and save them to an AWS S3 Bucket
The biggest drawback I encountered during development is that specifically for the Google Photos Library API, they don’t support the credential type: Service Account. Which makes the automation process more complicated in terms of avoiding human intervention. More about this here
This blog is made up of 2 parts:
• The first part is the core microservice which is responsible for downloading all the information from the Google Photos Library and storing it in the database and Amazon S3.
Communication between the client and the main service is through an MQTT broker (which I have not yet implemented) to notify the client of the photos to download based on different rules to follow:
• New photos in the Google Photos album will have priority
• Photos from the same day but different years ago will have the second priority as a way of looking back and memories. The metadata that Google Photos provides is really very poor from the point of view of social networks and there is not much that can be done in this regard.
All of these rules will be configured in the dispatcher microservice (not yet implemented).
Additionally, once this phase is over, I will be thinking of adding other media sources such as Facebook, Instagram, among others.
Steps to create a Google API project
From the dashboard click Create a new project.
Give it a name, and you’ll be redirected to the project dashboard:
Add an API by clicking Enable APIs and services.
From the list, search for photos Library API:
Create a new credential for the API
In the credentials section add a new credential type Oauth Client ID (Google Photos Library does not support services accounts)
Once you created it, you can download the credentials.json file by clicking:
Create a new Album in Google Photos
It is very simple to create an album in Google Photos. Here takes you to Google Photos and there you can create an album based on different types of filters.
For which I have created, it is an album of people that Google has identified through its Machine Learning. Which means that every time a photo of any of these people is uploaded, these photos will be added to the album automatically, therefore they will be shown in the virtual portrait automatically.
Run and Performance
In case you have concerns about pricing for the services used in this project.
For AWS S3 this is how it looks like for 20 GB of 2500 objects:
For me the ~2500 photos are using only 500 Mb of space. You can have an idea about pricing.
In the case of Google API. The limits and quotes are 10,000 requests per project per day.
The quota limit for requests to access media bytes (by loading a photo or video from a base URL) is 75,000 requests per project per day.
As usual, if you have any question, send me a message at email@example.com