Python と boto で S3 の情報を取得してみるメモ

# coding=utf-8
import json
import re

from boto3.session import Session

AWS_ACCESS_KEY_ID = 'AWS_ACCESS_KEY_ID...'
AWS_SECRET_ACCESS_KEY = 'AWS_SECRET_ACCESS_KEY...'
BUCKET_NAME = 'hoge'

session = Session(
    aws_access_key_id=AWS_ACCESS_KEY_ID,
    aws_secret_access_key=AWS_SECRET_ACCESS_KEY,
    region_name='ap-northeast-1',
)


# バケット名を表示してみる
s3 = session.resource('s3')
for bucket in s3.buckets.all():
    print(bucket.name)


s3client = session.client('s3')

response = s3client.list_objects(
    Bucket=BUCKET_NAME,
    Prefix='fuga_prefix'
)

keys = []
if 'Contents' in response:
    keys = [content['Key'] for content in response['Contents']]


# 特定の場所にあるjpeg画像を取得してJSONで書き出してみる
image_keys = []
for key in keys:
    if re.findall(r'^images/.*?\.jp[e]?g$', key, flags=re.I):
        image_keys.append(key)

records = {
    "Records": [{
        "bucket": BUCKET_NAME,
        "keys": image_keys
    }]
}

print(json.dumps(records, indent=2))

参考

• Boto 3 Documentation — Boto 3 Docs 1.2.3 documentation
• AWS - boto3 で デフォルトprofile以外を使う - Qiita

「git push ssh:〜」で「Permission denied (publickey).」とかなっちゃう件のメモ

失敗例

サーバ側（AWS Amazon Linux）

sudo mkdir /var/lib/git
sudo chown -R ec2-user /var/lib/git
cd /var/lib/git
mkdir demo.git
git init --bare demo.git

クライアント側

適当に何か作ってcommit

cd ~/projects/demo
echo 'hoge' > hoge.txt
git commit -m "first commit"
git init
git push ssh://ec2-xxx-xx-XX-xx.ap-northeast-1.compute.amazonaws.com/var/lib/git/demo.git master
    Permission denied (publickey).
    fatal: The remote end hung up unexpectedly

失敗となる

「Permission denied (publickey).」つうことなんで

クライアント側で

hogehost に秘密鍵を設定

秘密鍵が demo.pem の例

• ~/.ssh/config

Host hogehost
     HostName ec2-xxx-xx-xx-xx.ap-northeast-1.compute.amazonaws.com
     IdentityFile ~/.ssh/demo.pem
     User ec2-user

git push

git push ssh://hogehost/var/lib/git/demo.git master
    Counting objects: 20, done.
    Delta compression using up to 2 threads.
    Compressing objects: 100% (18/18), done.
    Writing objects: 100% (20/20), 9.45 KiB, done.
    Total 20 (delta 0), reused 0 (delta 0)
    To ssh://aar14b/var/lib/git/demo.git
     * [new branch]      master -> master

エラーでなくなった

git remote add origin ssh://hogehost/var/lib/git/demo.git
git push origin
    Everything up-to-date

でけた(・∀・)