how to combine data from different data sources?

  softwareengineering

I am trying to make a crypto analysis app. I fetch data from multiple platforms such as dexscreener, dextools, etc. The problem I run into is with poor code design/repetition and whenever I have to add new features or make changes, I have to make a lot of effort for refactoring.

The workflow is like this:

Get trendings from all the platforms, filter tokens(based on params say volume or market cap, or both), then fetch more details of filtered tokens such as ath, security status, socials, etc from different apis(because trending api doesn’t usually provide such info).

Now each platform has its API response structure, but I need specific set of information about a token such as(but not limited to):

Name 
Symbol
Social
Address
Price
Ath
MarketCap
Liquidity
SecurityScore
TopHolders
...

Now my approach is creating a mapper for each platform to extract related information from the API response, then I need to fill in null fields (say dexscreener has markerCap value, but dextools hasn’t for a token – by cross-checking) to complete the dataset values that I need, then it needs to merge/add further fields fetched from third-party apis (topHolders, SecurityScore, Taxes, …) to the existing dataset.

I want to design this code with minimum code repetition, maintainability, and flexibility to add new apis/features. How do I effectively tackle this problem? If you can also refer any resources on these topics that addressed these problems/solutions, would be highly appreciated. Thanks

New contributor

Kenny ripper is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.

LEAVE A COMMENT