Load external data: Google Drive, Sheets, and Cloud Storage

Tools & Scripts
1

a. Load data from local file system
Uploading files from the computer local file system

files.uploadreturn a dictionary of the files which were uploaded. The dictionary is keyed by the file name, the value is the data which was uploaded.
from
google.colab
import
files

uploaded = files.upload()

for
fn
in
uploaded.keys():
print(
‘User uploaded file “{name}” with length {length} bytes’
.format(name=fn, length=len(uploaded[fn])))

Resultado da execução: a sintaxe files.upload () fornece o upload de arquivos para o ambiente de execução da Colab

Downloading files to the computer local file system
files.downloadwill invoke a browser downloaded of the file to user’s local computer.
from
google.colab
import
files

with
open(
‘example.txt’
,
‘w’
)
as
f:
f.write(
‘some content’
)

files.download(
‘example.txt’
)

Resultado da execução: Abra o arquivo “example.txt” e escreva uma linha “algum conteúdo” durante a execução. Por fim, baixe o arquivo no seu próprio computador.

b. Load data from Google Drive

Access files in Google Drive using thenative REST APIor a wrapper likePyDrive.

PyDrive

The example below shows #1 authenticate, 2 file upload, and #3 file download from Google Drive. More examples are available in thePyDrive documentation

!pip install -U -q PyDrive

from

pydrive.auth

import

GoogleAuth

from

pydrive.drive

import

GoogleDrive

from

google.colab

import

auth

from

oauth2client.client

import

GoogleCredentials

1. Authenticate and create the PyDrive client.

auth.authenticate_user()

gauth = GoogleAuth()

gauth.credentials = GoogleCredentials.get_application_default()

drive = GoogleDrive(gauth)

PyDrive reference:

https://gsuitedevs.github.io/PyDrive/docs/build/html/index.html

2. Create

&

upload a file text file.

uploaded = drive.CreateFile({

‘title’

:

‘Sample upload.txt’

})

uploaded.SetContentString(

‘Sample upload file content’

)

uploaded.Upload()

print(

‘Uploaded file with ID {}’

.format(uploaded.get(

‘id’

)))

3. Load a file by ID and print its contents.

downloaded = drive.CreateFile({

‘id’

: uploaded.get(

‘id’

)})

print(

‘Downloaded content “{}”’

.format(downloaded.GetContentString()))

Resultado da execução: durante o processo de execução, a conta do Google a ser usada será selecionada, o código de autenticação da conta será obtido de acordo com as orientações e será colado no programa

Resultado da execução: gere um novo arquivo “Amostra upload.txt” no Google Drive

Resultado da execução: arquivo “Exemplo de upload.txt” no Google Drive

Drive REST API

Authentication is the first step

from

google.colab

import

auth

auth.authenticate_user()

Then construct a Drive API client.

from

googleapiclient.discovery

import

build

drive_service = build(

‘drive’

,

‘v3’

)

When client created, can use any of functions in theGoogle Drive API reference.

Create a new Google Drive file with data from Python

Create a local file.

with

open(

‘/tmp/to_upload.txt’

,

‘w’

)

as

f:

f.write(

‘my sample file’

)

print out the file content

print(

‘/tmp/to_upload.txt contains:’

)

!cat /tmp/to_upload.txt

Resultados da implementação:

After executing the code above, a new file named “Sample file” will appear in[drive.google drive file list.

from
 googleapiclient.discovery 
import
 build
drive_service = build(
'drive'
, 
'v3'
)


# Upload the file to Drive. See:
#
# https://developers.google.com/drive/v3/reference/files/create
# https://developers.google.com/drive/v3/web/manage-uploads
from
 googleapiclient.http 
import
 MediaFileUpload

file_metadata = {

'name'
: 
'Sample file'
,

'mimeType'
: 
'text/plain'

}
media = MediaFileUpload(
'/tmp/to_upload.txt'
, 
                        mimetype=
'text/plain'
,
                        resumable=
True
)
created = drive_service.files().create(body=file_metadata,
                                       media_body=media,
                                       fields=
'id'
).execute()
print(
'File ID: {}'
.format(created.get(
'id'
)))

The file ID in execution result will different with the example code above.

Resultado de execução

Downloading data from a Google Drive file into Python

Before execute the code below, change thefile_idfield by id with the file in Google Drive.

Download the file we just uploaded.

Replace the assignment below with your file ID

to download a different file.

A file ID looks like: 1uBtlaggVyWshwcyP6kEI-y_W3P8D26sz

file_id =

‘target_file_id’

import

io

from

googleapiclient.http

import

MediaIoBaseDownload

request = drive_service.files().get_media(fileId=file_id)

downloaded = io.BytesIO()

downloader = MediaIoBaseDownload(downloaded, request)

done =

False

while

done

is

False

:

_ is a placeholder for a progress object that we ignore.

(Our file is small, so we skip reporting progress.)

_, done = downloader.next_chunk()

downloaded.seek(

0

)

print(

‘Downloaded file contents are: {}’

.format(downloaded.read()))

Resultados da implementação:

c. Google Sheets
d. Google Cloud Storages (GCS)
Reference

2

Hi, sorry all
The links was working but I did not check them today, anyway, I already fixed dead links