Relative Content

Tag Archive for files

Is there a manageable way of keeping track of loose code? [closed]

It’s difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center. Closed 11 years ago. I program simple tasks everyday, but I have […]

Using assembly to write to a file

I am working with a trading application (reading data from the exchange) which generates a bucket load of data on a per second basis. We have different “log-levels” but even the minimal log-level generates so much data ! This process of log creation is quite I/O intensive.

Watching file changes/additions/removal, but with an eye on partial transfer

I would like to monitor the filesystem in python, so that my application gets warned of the new file addition, file removal, or file change. Once the file is detected, the application starts extracting the contained data through various plugins. The problem is that I am dealing with big files, and when the user starts copying a file from outside into the watched directory, it will be detected, but it will appear as corrupted. Checking for file size between invocations is potentially a good strategy, but it ignores the fact that other generators of the file (such as wget) might have long pauses when the file is not changing in size, and yet is not completed. I don’t have control of the file format I am downloading either, so I can’t check for an end-of-file mark, because it could not be there.

How are new file formats constructed?

I’ve used a software suite that is installed in offices and on remote vessels. The installations communicate back and forth, and they do that by using a simple proprietary file format that looks something like this: