How should I store an old project? [closed]

As a freelancer, I come across many projects. Its almost impossible to store a backup of all of the projects on my laptop. What are some better options to store them?

There are two sensible options, depending on the likelihood you’ll need to look at them again:

Burn the old projects to two DVDs, storing one “off site” somewhere
Storing them in the cloud on something like Dropbox

I have mine stored on USB flash drives. I keep a fireproof safe to store flash drives and ‘must’ have development CDs.

I’m a bit confused by your question:

Why are you storing backups on a laptop in a first place? Laptops are for work; backups are generally stored on dedicated servers or tapes or, for smaller companies, on external hard disks.
Why do you find it impossible? Because of the space it takes? What exactly is inside which takes so much place?

So, how should one store a backup of all projects? Here’s some guidelines.

Have dedicated hardware dealing with backups.

Ideally and if you have enough money, you would have a machine in your datacenter which will make regular scheduled backups of your version control. Depending on the size of the changes and the risk you can take, it may be scheduled as well to one backup per hour as to one backup per week.

In a less expensive environment, you’ll simply have an external hard drive you’ll plug regularly to do the backups on it. Beware of everything which requires manual effort or any manual intervention: while an automated process done exclusively by machines is mostly flawless, people asked to do the same task again and again can potentially forget doing it, or make a mistake or simply stop doing it, because they consider that they have “more important things to do than a backup” in time pressure context.
Always keep at least one backup off-site

There is small benefit in making backups if all those backups are stored in the same room or building. In case of fire, flood or anything which can affect the entire building, your data can be lost altogether with every backup you made.
Test your backup strategy

A few weeks ago, I was about to make a RAID configuration change on a NAS. The operation could potentially result in losing all the files. I had a not so bad backup strategy and data replication, and was sure it works pretty well, but I still decided to check twice if everything is fine. It appeared that a configuration change done a few months ago made the backup tool switching several backup destination paths to directories the backup tool wasn’t able to create. This meant that I could have lost more than three months of a part of important data.
Do incremental backups of version control

Version control can grow pretty fast if you work on large projects with lots of people involved. Dumping tens of thousands of revisions for several gigabytes of backup every hour or every day will be too expensive in terms of resources.

Most version control systems allow you to dump specific revisions. In my company, moving from daily entire dump to daily incremental dump allowed to considerably decrease the load on version control server, as well as waste more than 85% less space on backup machine (which also means wasting less network bandwidth for off-site backups).
Don’t delete your work

Some small companies with no money to buy new hardware are often tempted to delete some of the files involved in the original projects.

For example, a photographer may be asked to delete RAW files which may waste 30 MB or more space when he finished to work on a photograph and has the final JPEG.

Another example is to wipe out older versions from a version control, for example older than one year.

In both cases, the idea is bad, and can have important implications. For example, it would be difficult for a photographer to prove that a stolen photo was belonging to him if the only thing he has is the same JPEG that the stolen one. Wiping out older versions from version control erases statistical data which can be used in business intelligence.
Check what’s stored in your version control

Once, when I was auditing a company, I found that the version control was slow, fat, and totally unusable. Commits took forever, and desperate programmers preferred not doing any commit at all and share the source through the network directly. There were no backups, because it was practically impossible to do a dump, given the size of it.

After a bit of search, it appeared that every compiled file was in source control: every binary, every temp file, everything. This meant, if I remember well, that each commit was 30 MB in size. Indeed, this made it nearly impossible to do commits and increased the number of times programmers would go out to smoke or drink coffee.

Storing anything binary in an ordinary version control is an issue. This includes storing images or Word documents.

If your repository is fat, there may be something wrong in a way you use it. Check what’s inside, and react in consequence.

Well there are many ways to store backups but because the backups are not going to change over time and you might want to store them for a long time (probably >20 years). I would suggest that you use multipliable technologies to make backups, you never know if a technology will be unable with the dates technology in the future. I would recommend that you use at least 3 or more of this option. Store them in different places, like one at home, one at the office and so on in case of a fire or something else that would destroy the device. Also use different file system technologies, preferable at least one that is open source.

CD
DVD
Blu-ray
Hard disk drive
USB flash drive
Solid-state drive

I would not recommend using online backup like dropbox or any thing like this, you never know if company that is holding your data will go bankruptcy in the future. Also if you use a service like this the company have access to your source code and can do what the hell they want with it. You can use this as a complement to what i have mention before but i would strong suggest you encrypt the file before uploading them.

The technology changes too fast, there are definitely NOT any “forever” solutions that don’t involve you maintaining your backups. Keep multiple copies in different places and media. Today, flash drives and online backup look like good choices. As those good choices become less good and eventually die, keep moving on to new good choices. (My first set of archives were on paper tape. Second on 7 track mag tape.. you get the idea)

Long term archives should be in sealed checksummed packages, so you can at least
be sure they’re complete and uncorrupted.

Don’t overlook the importance of storing your archives with context. I guarantee that in 20 years, you won’t have a clue what most of the crap you saved was for; and worse, that if there’s some particular gem you’re looking for you’ll despair of locating it or using it once located.

Filed under: softwareengineering - @ 21:40

Thẻ: backups, freelancing

How should I store an old project? [closed]

As a freelancer, I come across many projects. Its almost impossible to store a backup of all of the projects on my laptop. What are some better options to store them?

There are two sensible options, depending on the likelihood you’ll need to look at them again:

Burn the old projects to two DVDs, storing one “off site” somewhere
Storing them in the cloud on something like Dropbox

I have mine stored on USB flash drives. I keep a fireproof safe to store flash drives and ‘must’ have development CDs.

I’m a bit confused by your question:

Why are you storing backups on a laptop in a first place? Laptops are for work; backups are generally stored on dedicated servers or tapes or, for smaller companies, on external hard disks.
Why do you find it impossible? Because of the space it takes? What exactly is inside which takes so much place?

So, how should one store a backup of all projects? Here’s some guidelines.

Have dedicated hardware dealing with backups.

Ideally and if you have enough money, you would have a machine in your datacenter which will make regular scheduled backups of your version control. Depending on the size of the changes and the risk you can take, it may be scheduled as well to one backup per hour as to one backup per week.

In a less expensive environment, you’ll simply have an external hard drive you’ll plug regularly to do the backups on it. Beware of everything which requires manual effort or any manual intervention: while an automated process done exclusively by machines is mostly flawless, people asked to do the same task again and again can potentially forget doing it, or make a mistake or simply stop doing it, because they consider that they have “more important things to do than a backup” in time pressure context.
Always keep at least one backup off-site

There is small benefit in making backups if all those backups are stored in the same room or building. In case of fire, flood or anything which can affect the entire building, your data can be lost altogether with every backup you made.
Test your backup strategy

A few weeks ago, I was about to make a RAID configuration change on a NAS. The operation could potentially result in losing all the files. I had a not so bad backup strategy and data replication, and was sure it works pretty well, but I still decided to check twice if everything is fine. It appeared that a configuration change done a few months ago made the backup tool switching several backup destination paths to directories the backup tool wasn’t able to create. This meant that I could have lost more than three months of a part of important data.
Do incremental backups of version control

Version control can grow pretty fast if you work on large projects with lots of people involved. Dumping tens of thousands of revisions for several gigabytes of backup every hour or every day will be too expensive in terms of resources.

Most version control systems allow you to dump specific revisions. In my company, moving from daily entire dump to daily incremental dump allowed to considerably decrease the load on version control server, as well as waste more than 85% less space on backup machine (which also means wasting less network bandwidth for off-site backups).
Don’t delete your work

Some small companies with no money to buy new hardware are often tempted to delete some of the files involved in the original projects.

For example, a photographer may be asked to delete RAW files which may waste 30 MB or more space when he finished to work on a photograph and has the final JPEG.

Another example is to wipe out older versions from a version control, for example older than one year.

In both cases, the idea is bad, and can have important implications. For example, it would be difficult for a photographer to prove that a stolen photo was belonging to him if the only thing he has is the same JPEG that the stolen one. Wiping out older versions from version control erases statistical data which can be used in business intelligence.
Check what’s stored in your version control

Once, when I was auditing a company, I found that the version control was slow, fat, and totally unusable. Commits took forever, and desperate programmers preferred not doing any commit at all and share the source through the network directly. There were no backups, because it was practically impossible to do a dump, given the size of it.

After a bit of search, it appeared that every compiled file was in source control: every binary, every temp file, everything. This meant, if I remember well, that each commit was 30 MB in size. Indeed, this made it nearly impossible to do commits and increased the number of times programmers would go out to smoke or drink coffee.

Storing anything binary in an ordinary version control is an issue. This includes storing images or Word documents.

If your repository is fat, there may be something wrong in a way you use it. Check what’s inside, and react in consequence.

CD
DVD
Blu-ray
Hard disk drive
USB flash drive
Solid-state drive

Long term archives should be in sealed checksummed packages, so you can at least
be sure they’re complete and uncorrupted.

Filed under: softwareengineering - @ 21:40

Thẻ: backups, freelancing

Thiết kế website giá rẻ

Danh mục

How should I store an old project? [closed]

How should I store an old project? [closed]