How to Implement Efficient Data Fetching between Client, API, and Database?

Recently, a client asked me to add a chart to the dashboard of one of their systems. The problem is that the dashboard was already huge, with several charts and tables, and it took forever to load. Everything was loaded before the page was rendered to the client, so the data appeared right away without any “loading” icons. This ended up being terrible for those who didn’t want to see that screen and wanted to do other tasks in the system.

When I first saw it, I thought it was absurd, but I didn’t know what to do to improve it.

After thinking more about it, I came to some conclusions about how to fetch data, but I still have some doubts:

Data fetching in the component:
This would work for quick data in components that aren’t a priority.
Example: a birthday table.
But in the example I gave, it wouldn’t work. Imagine around 12 components making requests to the API and querying the database at the same time, for each user who enters the system, right on the initial screen.
Data fetching (or real-time, depending on the case) from the parent component:
Useful when the data is the main reason for the page.
Example: a sales dashboard, competitions, or stock market data.
This could work since all the data would be fetched at once, but without locking the system for users who don’t need to see the dashboard information.
Load the data and wait for the entire page to load:
When the data is essential for the page to function.
Example: forms with data from an API.
This is what’s happening right now, but until all the data is loaded, no one can interact with the system, even those who don’t want to see the dashboard.
Fetch data on the first load and store it in the state:

Useful for small amounts of data that don’t take up much space on the client for future loads.
Example: forms with data from an API and frequently used data like dynamic buttons in navbars.
This doesn’t fit in this case because the data needs to refresh upon update, and there is too much data to keep in the state, which would take up a lot of RAM.

My doubts are:

Does what I said make sense?
Are there other options? If so, what are they and what are the pros and cons of each?
In all the available options like Vue3, Vanilla JS, JQuery, React, Angular, Blazor, and others, do these cases apply the same way?
What are the best practices and what should I avoid when fetching data (both in client-to-API communication and API-to-database, even though my post is mainly about Client-to-API)?
Is there any way to detect this type of performance issue before it becomes irritating, like what happened with this initial screen?
Do you have any stories to share about problems you, a colleague, or your company have faced due to poor data fetching management, and how it was resolved (client x API or API x database)?
Is there a place where I can learn more about this?

The system is built in Nuxt3. If there were examples, I think it would be easier to understand and learn what can be done.

Yes, your thoughts about data fetching and performance issues make sense, and you are on the right track. I’ll break down your points and add some options, best practices, and general insights on the topic.

1. Data Fetching in the Component (for non-priority data)

Use Case: Quick, low-priority data.
Issue: If too many components fetch data at the same time, this can overwhelm the API or block the UI.
Recommendation:

Lazy Loading: Load data when the user scrolls or interacts with a section.
Debouncing: If the data is not needed instantly, debounce the requests.

2. Fetching Data from the Parent Component (when the data is central)

Use Case: When the main focus of the page is on showing data (e.g., a dashboard).
Recommendation:

Batch Requests: Fetch data for multiple components in a single API call and distribute the results.
GraphQL: Use GraphQL for fetching multiple datasets in one request, reducing the number of API requests.
Priority Loading: Fetch critical data first, then load the rest asynchronously in the background.

3. Loading Data Before Page Rendering

Use Case: Data is essential for the page to function.
Issue: It leads to blocking the UI.
Recommendation:

Skeleton Loading or Spinners: Show a “skeleton screen” or loading spinner for the non-essential parts, allowing users to interact with the rest of the system while the data loads.
Server-Side Rendering (SSR): Nuxt3 supports SSR, and you can leverage this to load data before sending the page to the client. This makes initial loading faster, especially for critical data.

4. Fetch Once and Store in State

Use Case: Small amounts of data, like form options.
Recommendation:

Cache-Control Headers: Use HTTP cache headers on the API side so that frequently requested data can be cached on the client’s browser or intermediary caches (e.g., CDN).
Client-Side Caching: For small data that doesn’t change often, consider in-memory caching or using localStorage.

5. Additional Techniques and Options

SSR + CSR: Since you are using Nuxt3, leverage SSR for critical data and client-side rendering (CSR) for non-critical components to balance initial load time and responsiveness.
Code-Splitting: Lazy load components with data fetching only when they are visible or interacted with, reducing the upfront load.
Web Workers: For heavy data processing, offload the work to web workers to avoid blocking the main thread.
Pagination and Infinite Scroll: Don’t load all data at once; instead, load it in smaller chunks and append it as the user scrolls.

6. Is This Applicable Across Frameworks?

Yes, these techniques are mostly universal:

React, Vue, Angular: Support lazy loading, SSR, and client-side fetching. Nuxt3 is a good choice for SSR with Vue.
Vanilla JS, jQuery: Less automatic, but you can still implement lazy loading and fetch strategies.
Blazor: Similar strategies can be applied here, such as using lazy loading and managing API requests carefully.

The exact method of implementation varies, but the principles are consistent across most modern frameworks.

7. Best Practices for Data Fetching (Client-to-API)

Limit the Number of Requests: Try to reduce the number of API calls with batching and prioritization.
Cache Frequently Used Data: Use browser caching and client-side caching where applicable.
Lazy Load Non-Essential Data: Use lazy loading for non-critical content and heavy components.
Optimize API Requests: Use pagination, filtering, or GraphQL to fetch only the necessary data.

8. Detecting Performance Issues Early

Monitoring and Profiling Tools: Use tools like Google Lighthouse, Chrome DevTools, or Nuxt’s built-in analytics to monitor performance.
Performance Budgets: Set performance budgets (e.g., page load time, number of requests, etc.) and test regularly during development.
A/B Testing: Run tests with different data-fetching strategies to identify bottlenecks.
Log Latency in Your API: Measure how long the API is taking to respond and use tools like New Relic, Datadog, or custom logging to monitor for delays.

9. Example of Performance Issues I’ve Seen

Problem: In one project, a dashboard loaded every possible graph on page load, causing significant delays. Even for users who didn’t need to see the entire dashboard, the system was slow.
Solution: We used lazy loading for less critical components and prioritized key metrics first. We also implemented pagination for large datasets and refactored the backend API to use more efficient queries.

10. Resources for Learning More

Nuxt3 Documentation: Nuxt 3 Docs
Vue Performance Guide: Vue has excellent documentation on performance patterns.
Google Lighthouse: A great tool to check the performance of your application.
Udemy/Coursera: Look for courses on “Web Performance Optimization” or “Advanced Nuxt.js.”

Conclusion

You’ve identified good strategies already, and improving performance involves careful decisions about what data to load and when. I recommend focusing on lazy loading, batching requests, and SSR to improve your dashboard performance.

New contributor

Filed under: Kiến thức lập trình - @ 14:51

Thẻ: databaseclientnuxt3.js

Thiết kế website giá rẻ

Danh mục