Relative Content

Tag Archive for etl

Data warehouse architecture for mutating schema

I am setting up an ETL process and small data warehouse for querying the data in a few different dimensions. One issue is that the schema for the objects can mutate over time – mainly that some fields will be added and some removed. What are some methodologies or approaches for handling this?

Production or Custom Test Data for Unit Testing?

I’ve recently had a little disagreement with fellow developers. We’re transforming various ontologies from the original source format (Pica+, RDF, etc) into our data format and have several converters dedicated to exactly this task. The argument came whether or not we should use the full ontology-data file, or create a subset specifically used for testing.

Disagreement Concerning Data Integration (I may not understand enterprise ETL tools)

I have been in an ongoing conversation concerning a project we are about to undertake at my place at work. The project concerns data integration. Our customers want to be able to integrate our data with theirs, either by sending data from their systems into ours, or by getting data from our system into theirs. I don’t believe we are talking about ‘big data.’ Our entire database is less than 20 gigs.

Dynamically pass file name to FTP task in Realisable Iman

I want to generate a dynamic file name and pass it to the ‘Remote file’ fied of an FTP task in Realisable Iman. I have used a script task (Vb) to generate the file name but I can’t find away to pass that file name into the target field on the FTP Task.

Dynamically pass file name to FTP task in Realisable Iman

I want to generate a dynamic file name and pass it to the ‘Remote file’ fied of an FTP task in Realisable Iman. I have used a script task (Vb) to generate the file name but I can’t find away to pass that file name into the target field on the FTP Task.

Coordinating a complicated data migration process

A project I’m involved in has suffered a change in scope, and before I set about trying to cook up some homegrown solution, I’m wondering if there is something out there — some framework, for example — that will spare me from having to design and debug my own code. Let me try to explain the details as simply as possible.

Coordinating a complicated data migration process

A project I’m involved in has suffered a change in scope, and before I set about trying to cook up some homegrown solution, I’m wondering if there is something out there — some framework, for example — that will spare me from having to design and debug my own code. Let me try to explain the details as simply as possible.