{"id":2285,"date":"2021-02-08T15:28:33","date_gmt":"2021-02-08T12:28:33","guid":{"rendered":"https:\/\/artem.services\/?p=2141"},"modified":"2022-06-08T22:33:23","modified_gmt":"2022-06-08T19:33:23","slug":"2285","status":"publish","type":"post","link":"https:\/\/artem.services\/?p=2285&lang=en","title":{"rendered":"\u00a0Python &#8212; AWS S3 keep N latest artifacts"},"content":{"rendered":"<p><img loading=\"lazy\" class=\"aligncenter wp-image-2273 size-full\" src=\"https:\/\/artem.services\/wp-content\/uploads\/2021\/03\/Python-logo.png\" alt=\"\" width=\"1024\" height=\"500\" srcset=\"https:\/\/artem.services\/wp-content\/uploads\/2021\/03\/Python-logo.png 1024w, https:\/\/artem.services\/wp-content\/uploads\/2021\/03\/Python-logo-300x146.png 300w, https:\/\/artem.services\/wp-content\/uploads\/2021\/03\/Python-logo-768x375.png 768w, https:\/\/artem.services\/wp-content\/uploads\/2021\/03\/Python-logo-954x466.png 954w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/p>\n<p><span class=\"VIiyi\" lang=\"en\"><span class=\"JLqJ4b ChMk0b\" data-language-for-alternatives=\"en\" data-language-to-translate-into=\"ru\" data-phrase-index=\"0\" data-number-of-phrases=\"2\"><span class=\"Q4iAWc\">This script gets a list of all directories in the bucket and deletes all objects in each directory, except for the last &quot;<strong>N<\/strong>&quot; specified.<\/span><\/span> <span class=\"JLqJ4b ChMk0b\" data-language-for-alternatives=\"en\" data-language-to-translate-into=\"ru\" data-phrase-index=\"1\" data-number-of-phrases=\"2\"><span class=\"Q4iAWc\">To run the script, you need to pass two arguments:<\/span><\/span><\/span><\/p>\n<ul>\n<li>Bucket name<\/li>\n<li><span class=\"VIiyi\" lang=\"en\"><span class=\"JLqJ4b ChMk0b\" data-language-for-alternatives=\"en\" data-language-to-translate-into=\"ru\" data-phrase-index=\"0\" data-number-of-phrases=\"1\"><span class=\"Q4iAWc\">Number of last stored objects<\/span><\/span><\/span><\/li>\n<\/ul>\n<p>How to use it:<\/p>\n<pre class=\"brush: bash; title: ; notranslate\" title=\"\">\r\npython3 main.py artifacts-artem-services 3\r\n<\/pre>\n<p>&nbsp;<\/p>\n<h3>main.py:<\/h3>\n<pre class=\"brush: python; title: ; notranslate\" title=\"\">\r\nimport sys\r\nimport boto3\r\n\r\ndef cleanup():\r\n    get_last_modified = lambda obj: int(obj[&#039;LastModified&#039;].strftime(&#039;%s&#039;))\r\n\r\n    s3 = boto3.client(&#039;s3&#039;)\r\n    result = s3.list_objects(Bucket=bucket, Delimiter=&#039;\/&#039;)\r\n    for dir in result.get(&#039;CommonPrefixes&#039;):\r\n        print(&#039;Directory: &#039; + str(dir[&#039;Prefix&#039;]))\r\n        artifacts_listing = s3.list_objects_v2(Bucket = bucket, Prefix = dir.get(&#039;Prefix&#039;))[&#039;Contents&#039;]\r\n        artifacts_sorted = [obj[&#039;Key&#039;] for obj in sorted(artifacts_listing, key=get_last_modified)]\r\n        for artifact in artifacts_sorted[:-keep_last]:\r\n            print(&#039;Deleting artifact: &#039; + str(artifact))\r\n            s3.delete_object(Bucket = bucket, Key = artifact)\r\n\r\nif sys.argv[1:] and sys.argv[2:]:\r\n    bucket = sys.argv[1]\r\n    keep_last = int(sys.argv[2])\r\n    cleanup()\r\nelse:\r\n    print(&quot;This script for cleanup old artifacts in S3 bucket&quot;)\r\n    print(&quot;Usage  : python3 &quot; + sys.argv[0] + &quot; {BUCKET_NAME} &quot; + &quot;{NUMBER_OF_THE_LAST_KEEPING_ARTIFACTS}&quot;)\r\n    print(&quot;Example: python3 &quot; + sys.argv[0] + &quot; artifacts-artem-services &quot; + &quot;3&quot;)\r\n<\/pre>\n","protected":false},"excerpt":{"rendered":"<p>This script gets a list of all directories in the bucket and deletes all objects in each directory, except for the last &quot;N&quot; specified. To run the script, you need to pass two arguments: Bucket name Number of last stored objects How to use it: &nbsp; main.py:<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":[],"categories":[599],"tags":[543,1439,1527,1529,483],"_links":{"self":[{"href":"https:\/\/artem.services\/index.php?rest_route=\/wp\/v2\/posts\/2285"}],"collection":[{"href":"https:\/\/artem.services\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/artem.services\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/artem.services\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/artem.services\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2285"}],"version-history":[{"count":2,"href":"https:\/\/artem.services\/index.php?rest_route=\/wp\/v2\/posts\/2285\/revisions"}],"predecessor-version":[{"id":2288,"href":"https:\/\/artem.services\/index.php?rest_route=\/wp\/v2\/posts\/2285\/revisions\/2288"}],"wp:attachment":[{"href":"https:\/\/artem.services\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2285"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/artem.services\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2285"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/artem.services\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2285"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}