Questions tagged [amazon-s3]
Amazon S3 (Simple Storage Service) is a web service which provides storage and various interfaces for accessing data stored there.
174
questions
85
votes
7
answers
158k
views
Uploading files to S3 account from Linux command line
I've got several large files sitting in my Linux hosted account that I need to upload to my S3 account. I don't want to download them first and then upload them into S3.
Is there any way I can "...
- 2,777
61
votes
5
answers
28k
views
How to automate regular Google Takeout backups to cloud storage
I would like to create regular Google Takeout backups (let's say every 3 months) and store them encrypted in some other cloud storage like DropBox or S3.
It does not have to be a cloud-to-cloud ...
- 2,915
46
votes
8
answers
50k
views
Time machine backup on S3
Is there any possibility to use Amazon S3 for Mac OS X time machine backups?
Or in other words: Use S3 rather an external hard disk for time machine backups?
- 1,337
33
votes
5
answers
57k
views
23
votes
7
answers
19k
views
Replacing Dropbox with: Amazon S3 + SSL + GPG/TrueCrypt + Mounting on OSX ?? [closed]
So, right now we're using Dropbox to share various data files around between approximately 10 Mac OS X systems.
However, we already have an S3 account and everyone on the lowest Dropbox plan of $10/...
- 775
22
votes
5
answers
33k
views
Can I use rsync for backing up at Amazon S3
I have bought some space from evbackup which provides me FTP, rsync and SSH.
But I have also read that Amazon S3 can also be used to backups.
So I am really confused that do Amazon give FTP access ...
- 2,943
21
votes
5
answers
32k
views
A decent S3 bucket manager for Ubuntu
I'm looking for a decent S3 bucket manager for Ubuntu (Gnome). I prefer it to integrate with Nautilus so it will look like just any other drive (a la WebDAV) but so far I haven't been able to find ...
- 2,649
19
votes
1
answer
38k
views
Amazon S3 GUI client for OS X [closed]
I'm looking for a good GUI client for Amazon S3. On Windows I used Cloudberry Explorer and I would like to know if there's anything like that for Mac OS X.
- 735
16
votes
2
answers
19k
views
AWS upload folder to S3 as tar.gz without compressing locally
In AWS CLI, how do I upload a folder as a tar.gz file without creating a tar.gz locally?
For example, I have a folder at /var/test and I want to upload it to /tests/test1.tar.gz
How do I do that ...
- 161
10
votes
6
answers
12k
views
Alternative to Amazon's S3 service? [closed]
Just wondering if there is good alternative to Amazon's S3 service? I like S3 but the bandwidth cost is high. I looked at CouldFiles from Rackspace but the cost is even higher.
I don't mind ...
- 169
10
votes
3
answers
11k
views
How to upload big files to S3 on a flaky connection?
I have a bunch of files (between 500MB and 7GB) that I need uploaded to an S3 bucket. My connection is very flaky.
I tried uploading a 500MB file via s3cmd but it timed out after it was 91% done (...
- 663
9
votes
2
answers
12k
views
Auto-set permissions when uploading to Amazon S3 with Transmit 4 [closed]
Does anyone know how to auto-set the read permissions for "World" upon upload of a file to Amazon S3 with Transmit? It is currently a two step process in my workflow:
upload the file
change ...
- 273
9
votes
5
answers
3k
views
Let's Encrypt with Amazon S3
Will it be possible to have https on the my static Amazon S3 website with Let's Encrypt?
Even I have Let's Encrypt Closed Beta Invite now. But how to use it with Amazon S3?
- 111
7
votes
1
answer
6k
views
Know the identity behind an AWS canonical ID
Today while auditing a friends security settings, I found that an Amazon Folder (s3 bucket) with his most personal files had an unrecognised ACL entry.
A person (with id ending in f6995f) had "Write" ...
7
votes
0
answers
17k
views
Can't connect to S3 using cyberduck
I'm trying to connect to my S3 bucket using cyberduck.
I select S3 from the dropdown.
Input into server name: {bucket-name}.s3.amazonaws.com
Access key ID and Secret Access Key into the next two ...
- 171
6
votes
1
answer
21k
views
GPG program not found on OS X
Trying to run s3cmd on my mac book air, receiving the following message.
Success. Your access key and secret key worked fine :-)
Now verifying that encryption works... ERROR: Test failed: GPG ...
- 421
6
votes
1
answer
12k
views
Why can't I connect to my Amazon S3 storage with two of my three clients?
I use Transmit, Cyberduck, and Espresso, all of which support connecting to Amazon S3. But for some reason, I can only connect to my S3 storage successfully in Transmit; Cyberduck and Espresso both ...
- 135
6
votes
0
answers
2k
views
AWS Client S3 mv command is slow
I am using aws cli to move the files around in S3 in the same bucket. As far as I know, AWS S3 does not maintain the folder structure. It stores the entire path as key to the file and displays it in ...
- 123
6
votes
3
answers
4k
views
What free method would you recommend for mounting an Amazon S3 bucket on OS X 10.6?
It seems s3fs is broken on Snow Leopard
( http://mark.aufflick.com/blog/2007/10/28/leopard-amazon-s3-network-storage )
Does anyone know of a free (and preferably open source) way to mount a bucket so ...
- 9,883
5
votes
2
answers
10k
views
How to search for nested files in my S3 bucket?
I'm on the S3 Management Console and I see the instructions to type a prefix to search:
However, when I search, it only finds files in the current folder. Is there a way to search for files that ...
- 157
5
votes
1
answer
4k
views
How to mount Amazon S3 bucket to a directory on FreeBSD?
I can't find a solution on how to mount an Amazon S3 bucket to an existing directory on FreeBSD.
I suspect, I need to install port fusefs-s3fs.
But how should I configure it? I can't find any info.
- 175
4
votes
4
answers
2k
views
Can I rely on S3 to keep my data secure?
I want to back up sensitive personal data to S3 via an rsync-style interface. I'm currently using s3cmd - a great tool - but it doesn't yet support encrypted syncs. This means that while my data is ...
- 143
4
votes
1
answer
4k
views
How can I download a copy of an S3 public data set?
i was naively assuming I could do something like
s3cmd sync s3://snap-d203feb5 /var/tmp/copy
but I seem to have the wrong idea of how to go about this. I cannot even get a simple thing to work;
...
- 3,231
4
votes
3
answers
3k
views
Transfer file to Amazon S3 from web server
How do I transfer a tarball from my web server (e.g. http://example.com/backup.tar.gz) to an Amazon S3 bucket? Do I have to download it locally and upload it?
- 43
4
votes
1
answer
5k
views
multiple s3 users in one bucket or per-user buckers
intelligent people, noob here!
I am planning on building a multi-user apparatus on s3 for photo/object storage, and I was planning on using s3. I have the whole front-end planned out but I have a ...
- 141
4
votes
1
answer
564
views
A company wide dropbox like system on top of Amazon S3
Is there a system that will provide dropbox like features on top of Amazon s3?
It would be used company wide, mostly for documents, recordings, etc...
Some users work on Microsoft Windows, others on ...
- 1,003
4
votes
5
answers
863
views
Cloud based backup solutions based on open standards?
I am looking for a solution to backup and consolidate important media from a couple Windows laptops and Mac laptop.
I would like a solutions that based on open standards, so my data isn't trapped by ...
- 141
4
votes
1
answer
3k
views
S3 based dropbox like program for mac os and windows [closed]
I use dropbox to sync file between several computers. It does the job quite well but it have several disadvantages:
Storing files on S3 is much cheaper (I need about 10Gb for all my shared files) and ...
4
votes
1
answer
434
views
Automatically Transcode Video and Upload to Cloud
I'm looking for a solution to take video files from my HD Camera and automate transcoding to a smaller format and then uploading to the cloud (AWS or otherwise).
The ideal workflow would be, I have a ...
- 285
3
votes
5
answers
2k
views
Setting up scripts in Amazon EC2 Cloud
I am currently running a few perl and python scripts on a windows pc and would like to port over to the Amazon EC2 servers running 64-bit LINUX. The scripts are basic web scrapers that go to a ...
- 131
3
votes
2
answers
7k
views
Using S3 instead of FTP- any reason NOT to?
Whenever a client wants an FTP storage solution, my instinct it to direct them to S3 instead, as a secure and cost-effective alternative. Is there any good reason NOT to use S3 instead of FTP/SFTP in ...
- 406
3
votes
2
answers
560
views
Amazon S3 - To Backup or Not to Backup
So heres a question: Should I back up my data if I store using S3 (not RRS) or does S3 already handle a good enough multiple site data replication redundancy?
If backing up is desired, I'd like to ...
- 133
3
votes
1
answer
2k
views
Encryption strategy for Amazon S3?
I'm running Windows and strongly considering using Amazon S3 to back up some of my more important data, but i'm having trouble coming up with a good encryption strategy. I'm a fan of Truecypt and use ...
- 31
3
votes
1
answer
1k
views
Nginx SSL Preread Sporadically Gets Requests Where No Server Name Is Extracted
I am using the nginx stream module to leverage nginx as a tcp reverse proxy in front of s3. Recently, I needed to add logic to accommodate an additional upstream. To do this, I chose to make use of ...
- 143
3
votes
1
answer
2k
views
AWS S3 cp does not download 40 GB file
I just uploaded a 40GB backup file from my linux EC2 machine to an S3 bucket. However when I want to download it the aws command ends with no message and nothing was copied.
Here is my command:
...
- 193
3
votes
1
answer
1k
views
Are whole VM images backed up on Amazon EC2/S3?
I've been trying to get my head around Amazon Web Services as a VPS provider. My understanding is a EC2 instance running Windows is basically a Windows VM, very similar to renting a VPS from a more ...
- 5,316
3
votes
1
answer
2k
views
https proxy s3 aws via nginx with pagespeed
I'm trying to setup pagespeed to proxy s3 files via nginx. I want to "save" the files on amazon s3;
The story I want is:
User A
1. nginx serves page to user; and rewrites all https://s3.amazonaws....
- 163
3
votes
1
answer
2k
views
Use personal Amazon S3 account to backup Gmail using Ubuntu
As question, I have found online that backupify offer shifting from their S3 to user's personal S3, I have a backupify account but I can't find this options, besides, I don't prefer having my email ...
- 523
3
votes
0
answers
369
views
How to use AWS S3 for linux file system backup?
As many who have used AWS S3 realize, it isn't a file system, but everything in the bucket is an object. While using the aws cli to do backups of a Linux server to S3, it doesn't preserve the ...
- 557
3
votes
0
answers
717
views
s3fs gives me an empty directory
I built s3fs, and now I'm getting an empty directory where I mounted the s3 bucket. I put a text file in the directory, and I can see it in there, and it transfers into the bucket. How can I see the ...
- 211
3
votes
1
answer
2k
views
Subversion repository on Amazon S3
Is it possible to put a subversion repository on Amazon S3.
I found SvnToS3 on google code but that project seems dead.
- 41
2
votes
1
answer
1k
views
Is having Linux VPS same as Amazon EC2 or S3 [closed]
I am new to Amazon services so have no idea, whenever i ask some questions on forums , few recommend Amazon instead of VPS.
So i want to know is having Amazon EC2 or S3 is exact replacement of VPS. i ...
- 2,943
2
votes
2
answers
864
views
Why CNAME is not working as supposed?
I have set a CNAME entry to redirect over Amazon S3 bucket, however there is something that doesnt works as supposed (by me of course) with CNAME I expect to redirect a fictional subdomain, say for ...
- 83
2
votes
4
answers
2k
views
What desktop app can sync Amazon S3 with an FTP folder?
What I'm trying to do is migrate large files off my web site to S3.
- 1,059
2
votes
2
answers
17k
views
curl can not download file but browser can
URL example: https://xxxxxxxxxx.s3.amazonaws.com/xxxx-xxxx-xxxx-xxxx/xxxxxxxxxxxxx/x?AWSAccessKeyId=AKIAIFKEBR4CLVTDYSLA&Signature=Kv%2B859dJ2eUmLgHdkT%2F9nUJEeRM%3D&Expires=1922517562
I ...
- 190
2
votes
1
answer
7k
views
Determining the service that is using *.CloudFront.net
How can I determine that accessing to cloudfront.net is associated to which service, software or owner?
(as far as I understand and the normal who-is lookup tells, it is related to Amazons' cloud ...
- 182
2
votes
1
answer
1k
views
s3cmd tool falls over on local directory
I'm trying to use s3cmd to copy a full bucket down to my local machine like so:
s3cmd -c .s3cfg-bespoke sync s3://my_bucket/ ~/local_folder/
There are three folders and a smattering of files in the ...
- 385
2
votes
1
answer
877
views
Zip Multiple Text Files from Multiple Directories into One ZIP File in One Command in Bash
I have an account on AWS and I use S3 for storing data. I have commands that will enable me to zip data files like so:
data.zip s3_dir/level-1/level-2/*/*/data.txt
But there are, say, 50 or more ...
- 6,725
2
votes
2
answers
6k
views
copy specific files from s3 bucket
I have some files in s3 bucket.
aws s3 ls s3://bucketname/
file-111-100x100.jpg
file-112-1400x1400.jpg
file-123-250x250.jpg
file-231-1400x1400.jpg
file-222-700x700.jpg
file-333-100x100.jpg
file-131-...
- 63
2
votes
1
answer
292
views
Backup software for Ubuntu - which one? [closed]
I have spent some time testing out different backup solutions for my small home office during the last weeks, but still haven't found anything that have been working out too well yet. We can ...
- 196