Cycling and bicycle discussion forums. 
   Click here to join our community Log in to access your Control Panel  


Go Back   > >

Foo Off-Topic chit chat with no general subject.

User Tag List

Reply
 
Thread Tools Search this Thread
Old 11-02-06, 04:00 PM   #1
Jerseysbest
J E R S E Y S B E S T
Thread Starter
 
Jerseysbest's Avatar
 
Join Date: Apr 2005
Location: DC
Bikes:
Posts: 1,849
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
What's the most efficient way to transfer data?

Something came up at work and we're wondering if we could, on about a weekly basis, transfer about 1.0 to 1.3 GB of data from one computer to about 5, or as many as 10, others, all of which are in different locations.

Do you think this could be done in a reasonable amount of time? I guess the cheapest solution would be USB 2.0 flash drives or external hard drives?

Anyone have experience doing this kind of stuff?
Jerseysbest is offline   Reply With Quote
Old 11-02-06, 04:03 PM   #2
jschen
riding once again
 
jschen's Avatar
 
Join Date: Oct 2005
Location: San Diego, CA
Bikes: '06 Cervelo R3, '05 Specialized Allez
Posts: 7,359
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Efficient for the computer or efficient for the human? Different locations in the same building or geographically separated? If geographically separated, is the data sensitive? Why not just make a bunch of copies via file sharing? Sure, if your network and/or internet connection is slow it might take a while, but you can initiate the process at the end of a workday and just let it go.
__________________
If you notice this notice then you will notice that this notice is not worth noticing.
jschen is offline   Reply With Quote
Old 11-02-06, 04:09 PM   #3
TexasGuy
That darn Yankee
 
TexasGuy's Avatar
 
Join Date: Jun 2005
Location: West West Fort Worth
Bikes: Mongoose XR-100, Eros Bianchi
Posts: 4,286
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
I'd try using a network. Network should be tons times faster then USB and much less processor intensive. A 1gb file can transfer quite fast over a modern network (preferably wired)
__________________
Life is about hanging onto what you think is important and finding out what really is important.
"Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
So many sayings, so little sig space.
TexasGuy is offline   Reply With Quote
Old 11-02-06, 04:12 PM   #4
Glenfiddich_Man
Junior Member
 
Glenfiddich_Man's Avatar
 
Join Date: Sep 2006
Bikes:
Posts: 9
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Pretty much passing around a flash drive would be the quickest/cheapest

http://www.amazon.com/SanDisk-Cruzer...&s=electronics
Glenfiddich_Man is offline   Reply With Quote
Old 11-02-06, 04:21 PM   #5
TexasGuy
That darn Yankee
 
TexasGuy's Avatar
 
Join Date: Jun 2005
Location: West West Fort Worth
Bikes: Mongoose XR-100, Eros Bianchi
Posts: 4,286
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Flash drives are slow. The fast ones are expensive and they have a limited # of writes.
I would never use a Flash Disk system for something like this. Unless there was no way to run network cables or run wireless as wireless should be faster then USB, is not limited by write speed and does not wear down after x writes.
__________________
Life is about hanging onto what you think is important and finding out what really is important.
"Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
So many sayings, so little sig space.
TexasGuy is offline   Reply With Quote
Old 11-02-06, 04:31 PM   #6
TRaffic Jammer
Dances With Cars
 
TRaffic Jammer's Avatar
 
Join Date: Jan 2006
Location: Toronto, Canada
Bikes: TBL Onyx Pro(ss converted), Pake SS (starting to look kinda pimped)
Posts: 10,527
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
When you say different locations...different cities?
Publish once to FTP/ or intranet site (if offices are linked with one)...let branch office download at their leisure. Or write to their respective sites at the same time (overnight lan admin operation). You network administrator will have your solution where they write to the locations all at once.

If it's within the same building/wired network, simply post to network volume and it's accessible.

If it's Sensitive with capitol S burn and ship DVD's
TRaffic Jammer is offline   Reply With Quote
Old 11-02-06, 04:39 PM   #7
Jerseysbest
J E R S E Y S B E S T
Thread Starter
 
Jerseysbest's Avatar
 
Join Date: Apr 2005
Location: DC
Bikes:
Posts: 1,849
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
All the computers are in different geographic locations, meaning, not in the same building.

I thought about the internet, but I'd guess that the max download speed would be 1 mb/sec, so that would take forever, and since we'd like to not have the data get into anyone else's hands.

One other idea I thought about are burning it onto DVD's. Have the people who will be bringing the data to the remote machines get the data off our network, and the burn it. For something that would happen on a weekly basis, I guess it could work.
Jerseysbest is offline   Reply With Quote
Old 11-02-06, 04:48 PM   #8
jschen
riding once again
 
jschen's Avatar
 
Join Date: Oct 2005
Location: San Diego, CA
Bikes: '06 Cervelo R3, '05 Specialized Allez
Posts: 7,359
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Proper encryption (which would be necessary even with the DVDs) would help protect sensitive information. So I don't see going over the Internet via some sort of VPN as being a problem. I know of companies that transfer huge amounts of extremely sensitive data that way. As for time, well, at 1 MB/s, you'd copy those files in about 20 minutes. Let's say it takes 30 minutes. You really think you could burn a DVD, drive it over, and then upload it faster than that? And the electronic transfer takes far less work. Even if you just make transfers all from the same computer (you could do it faster if you script it so that after the first transfer, that computer starts transferring, too), and it takes 30 minutes per transfer, that only takes 5 hours max. Even if it takes an hour per transfer, that's 10 hours. Surely you have a 10 hour window over the weekend? And importantly, you don't have to be there to get the job done. It could be scripted to run automatically. Or you could just start the transfers at the end of the work week and let them go.
__________________
If you notice this notice then you will notice that this notice is not worth noticing.
jschen is offline   Reply With Quote
Old 11-02-06, 04:49 PM   #9
TRaffic Jammer
Dances With Cars
 
TRaffic Jammer's Avatar
 
Join Date: Jan 2006
Location: Toronto, Canada
Bikes: TBL Onyx Pro(ss converted), Pake SS (starting to look kinda pimped)
Posts: 10,527
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
you can get blistering fast data transfers... I have three fibre lines to our press centre at the other end of the city, but it's full all day everyday. A weekly burn and ship seems fair inexpensive in comparison
TRaffic Jammer is offline   Reply With Quote
Old 11-02-06, 05:01 PM   #10
Portis
Banned.
 
Join Date: Sep 2003
Location: Home alone
Bikes: Trek 4300 X 2. Trek 1000, Trek 6000
Posts: 6,019
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Dvd Ram
Portis is offline   Reply With Quote
Old 11-02-06, 05:37 PM   #11
DannoXYZ 
Senior Member
 
DannoXYZ's Avatar
 
Join Date: Jul 2005
Location: Saratoga, CA
Bikes:
Posts: 11,600
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Why aren't all your buildings on a network? Sneaker-net went out of style decades ago... You can set up a private WiFi network using directional antennaes with ranges of about a mile.

jschen's got the right idea. It's simplest to use the network someone else has developed. Saves you time, effort and money. You can even set up a sftp-server and have the files posted for all them to download. Encryption will ensure security. Compression can probably crunch that gig of data down quite a bit depending upon what type of data it is.

If it's something that you do on a regular basis, you can even write a script that'll upload to the server and automatically download to each of the client stations.
DannoXYZ is offline   Reply With Quote
Old 11-02-06, 05:43 PM   #12
KingTermite 
On my TARDIScycle!
 
KingTermite's Avatar
 
Join Date: Jun 2005
Location: Eastside Seattlite Termite Mound
Bikes: Trek 520, Trek Navigator 300, Peugeot Versailles PE10DE
Posts: 3,924
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Quote:
Originally Posted by TexasGuy
I'd try using a network. Network should be tons times faster then USB and much less processor intensive. A 1gb file can transfer quite fast over a modern network (preferably wired)
+1

If its the same data, you could even automate it with a batch file or script.
__________________
Quote:
Originally Posted by coffeecake View Post
- it's pretty well established that Hitler was an *******.
KingTermite is offline   Reply With Quote
Old 11-02-06, 06:28 PM   #13
TexasGuy
That darn Yankee
 
TexasGuy's Avatar
 
Join Date: Jun 2005
Location: West West Fort Worth
Bikes: Mongoose XR-100, Eros Bianchi
Posts: 4,286
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Quote:
Originally Posted by TRaffic Jammer
you can get blistering fast data transfers... I have three fibre lines to our press centre at the other end of the city, but it's full all day everyday. A weekly burn and ship seems fair inexpensive in comparison
I wonder why printing press and likees networks are always bogged down
__________________
Life is about hanging onto what you think is important and finding out what really is important.
"Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
So many sayings, so little sig space.
TexasGuy is offline   Reply With Quote
Old 11-02-06, 06:48 PM   #14
TRaffic Jammer
Dances With Cars
 
TRaffic Jammer's Avatar
 
Join Date: Jan 2006
Location: Toronto, Canada
Bikes: TBL Onyx Pro(ss converted), Pake SS (starting to look kinda pimped)
Posts: 10,527
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
^^ NOT that our ISP runs through those channels, TG smar-ty
TRaffic Jammer is offline   Reply With Quote
Old 11-02-06, 06:53 PM   #15
russiankdi
Senior Member
 
russiankdi's Avatar
 
Join Date: Feb 2006
Location: PA
Bikes: many
Posts: 213
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
If all the computers are hooked up to one main network, you can setup up so that all the computers can be shared, that way files can be transfered from one computer to as many as you want. On my home network i have 3 computers hooked up, and i can transfer data really easily, up to 100mbp/s.
__________________
1978 Schwinn Varsity Single Speed
russiankdi is offline   Reply With Quote
Old 11-02-06, 07:00 PM   #16
TexasGuy
That darn Yankee
 
TexasGuy's Avatar
 
Join Date: Jun 2005
Location: West West Fort Worth
Bikes: Mongoose XR-100, Eros Bianchi
Posts: 4,286
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Quote:
Originally Posted by russiankdi
If all the computers are hooked up to one main network, you can setup up so that all the computers can be shared, that way files can be transfered from one computer to as many as you want. On my home network i have 3 computers hooked up, and i can transfer data really easily, up to 100mbp/s.
that aspect of the problem has already been delineated. They're not. They are in different geogrpahical locations.
__________________
Life is about hanging onto what you think is important and finding out what really is important.
"Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
So many sayings, so little sig space.
TexasGuy is offline   Reply With Quote
Old 11-02-06, 07:14 PM   #17
TRACKMAN
hide not your essence
 
TRACKMAN's Avatar
 
Join Date: Oct 2006
Location: Dodge City
Bikes: Airborne,Guerciotti,Jamis,Merckx,Paris Sport,Raleigh,Vitus
Posts: 155
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
you could do a point to point; some browsers, i.e. DEEPNET enable this and you can set passwords.
FTP is archaic and slower, and an open share on a network means ANYONE that can see it can snatch it.
of course, you could set passwords on that share as well but throughput would degregate depending on amount of traffic in that subnet. Not to mention if you were leaving that net you would be subjugated to the router or switches ability to provide bandwith as well. Regardless of how much bandwidth you have, you are still limited by the slowest hop along the path between two nodes.
__________________
May you find the joy and peace you long for.
Life is a journey ... NOT a guided tour.
.......__O
.......\<,
....( )/ ( )...
TRACKMAN is offline   Reply With Quote
Old 11-02-06, 07:17 PM   #18
jschen
riding once again
 
jschen's Avatar
 
Join Date: Oct 2005
Location: San Diego, CA
Bikes: '06 Cervelo R3, '05 Specialized Allez
Posts: 7,359
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Passwords alone don't do you much good if you're transmitting unencrypted data in the clear. If the data is at all sensitive, encryption is necessary. Whether you do this at the file level or at the network level is up to you. I'd do it at the network level to avoid having to encrypt and decrypt files over and over.
__________________
If you notice this notice then you will notice that this notice is not worth noticing.
jschen is offline   Reply With Quote
Old 11-02-06, 07:20 PM   #19
Jerseysbest
J E R S E Y S B E S T
Thread Starter
 
Jerseysbest's Avatar
 
Join Date: Apr 2005
Location: DC
Bikes:
Posts: 1,849
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Thanks for all the input, I'm still trying to wrap my head around this and figure something out.

There are a lot of variables, so I don't know if we'll be able to afford the time of downloading over the net, but that option is on the table.

Because the person that will be going to the remote machine already has their own machine here in the office with a DVD burner, is on the network, and a batch file set up to retrieve the data from the main machine, I'm leaning towards having each person burn the data onto the DVD.

This might all be done sorta on the fly, so weekend downloads probably aren't an option, and I think we're already getting flack for our bandwidth use.
Jerseysbest is offline   Reply With Quote
Old 11-02-06, 07:28 PM   #20
jschen
riding once again
 
jschen's Avatar
 
Join Date: Oct 2005
Location: San Diego, CA
Bikes: '06 Cervelo R3, '05 Specialized Allez
Posts: 7,359
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
How far are the buildings? Transfering data by shuffling people back and forth is truly inefficient. If these transfers are made "on the fly", how urgent is the data?

If you must shuffle people, I'd go with portable hard drives (Firewire or USB2). Having lots of DVDs sitting around with sensitive data is potentially problematic. You'd have to ensure each of those DVDs is destroyed after use. And hard drive transfers are far faster.
__________________
If you notice this notice then you will notice that this notice is not worth noticing.
jschen is offline   Reply With Quote
Old 11-02-06, 08:03 PM   #21
TexasGuy
That darn Yankee
 
TexasGuy's Avatar
 
Join Date: Jun 2005
Location: West West Fort Worth
Bikes: Mongoose XR-100, Eros Bianchi
Posts: 4,286
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Quote:
Originally Posted by jschen
How far are the buildings? Transfering data by shuffling people back and forth is truly inefficient. If these transfers are made "on the fly", how urgent is the data?

If you must shuffle people, I'd go with portable hard drives (Firewire or USB2). Having lots of DVDs sitting around with sensitive data is potentially problematic. You'd have to ensure each of those DVDs is destroyed after use. And hard drive transfers are far faster.
The same issue happens with USB also. Some janitor picks up usb, takes it home . bam. Information is compromised. I'd be tempted -

if they are going from laptop to computer why not just use a wireless network.
__________________
Life is about hanging onto what you think is important and finding out what really is important.
"Stop Ruining my joke!", "No, a joke implies humor attached at no additional cost"
So many sayings, so little sig space.
TexasGuy is offline   Reply With Quote
Old 11-02-06, 08:56 PM   #22
Jerseysbest
J E R S E Y S B E S T
Thread Starter
 
Jerseysbest's Avatar
 
Join Date: Apr 2005
Location: DC
Bikes:
Posts: 1,849
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Quote:
Originally Posted by jschen
How far are the buildings? Transfering data by shuffling people back and forth is truly inefficient. If these transfers are made "on the fly", how urgent is the data?

If you must shuffle people, I'd go with portable hard drives (Firewire or USB2). Having lots of DVDs sitting around with sensitive data is potentially problematic. You'd have to ensure each of those DVDs is destroyed after use. And hard drive transfers are far faster.
Well, I'd assume we'd destroy the data DVD once it was transfered, and the cost isn't prohibitive.

And the the other computers are too far to utilitize a wireless network.
Jerseysbest is offline   Reply With Quote
Old 11-02-06, 09:03 PM   #23
SaabFan
The AVatar Ninja
 
SaabFan's Avatar
 
Join Date: Aug 2006
Bikes:
Posts: 609
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
How about a shared workspace application like Groove?

Install it on each machine, leave them connected to the internet and logged in to a shared Groove workspace. The files in the workspace are automatically distributed to each computer over the internet, encrypted, in the background. No human interaction required. It'll even do automatic backups of everything, and lets you use version tracking if the files are changed over time by multiple users.
SaabFan is offline   Reply With Quote
Old 11-02-06, 09:09 PM   #24
russiankdi
Senior Member
 
russiankdi's Avatar
 
Join Date: Feb 2006
Location: PA
Bikes: many
Posts: 213
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
Quote:
Originally Posted by TexasGuy
that aspect of the problem has already been delineated. They're not. They are in different geogrpahical locations.
In that case external USB hard drive, have one of those as well, works great.
__________________
1978 Schwinn Varsity Single Speed
russiankdi is offline   Reply With Quote
Old 11-02-06, 09:28 PM   #25
khuon
DEADBEEF
 
khuon's Avatar
 
Join Date: Aug 2002
Location: Catching his breath alongside a road near Seattle, WA USA
Bikes: 1999 K2 OzM, 2001 Aegis Aro Svelte
Posts: 12,242
Mentioned: 0 Post(s)
Tagged: 0 Thread(s)
Quoted: 0 Post(s)
I think everyone forgot to ask the critical question here...

What's your budget?

Also: What's the distance? Can the data be streamed and/or updated incrimentally? Is it one big file or multiple files? What's the nature of the data? Text? Binary?
__________________
1999 K2 OzM 2001 Aegis Aro Svelte
"Be liberal in what you accept, and conservative in what you send." -- Jon Postel, RFC1122
khuon is offline   Reply With Quote
Reply


Thread Tools Search this Thread
Search this Thread:

Advanced Search

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off



All times are GMT -6. The time now is 02:39 AM.