Modern architectures - no-code solution for transforming data into actionable insights

Technologies

C#, ASP.NET, Angular, NATS, Elasticsearch, Quartz.NET, MediatR, EPPlus, Entity Framework Core, PostgreeSql

CATEGORY

Web application development

Home > Webapp For Dynamic Reports

The purpose of the project was to create a web application that allows the generation of dynamic reports and customized graphical visualizations, using both external and local data.

Challenges

The client needed a solution that offered maximum flexibility in data manipulation and visualization without compromising ease of use. The main challenges were: 
  • Generating dynamic reports with multiple and configurable data sources.
  • Advanced customization and configuration of dynamic charts.
  • The need to have real-time updated data for a seamless experience. 
  • Exporting both calculations and formulas to Excel format, allowing the user to manipulate and simulate with the resulting data. 
  • Ensuring high performance and efficient management of large data volumes. 

Solution

The developed solution aims to provide an intuitive platform capable of meeting the requirements for flexibility in data management, interactivity in data visualization, and easy export of results into a wide range of formats. 


The project is a web application developed using C# (ASP.NET Core) for the back-end and Angular for the front-end. 

Roles Covered

1Project manager
1 Project Architect
1Developer

Solutions

Dynamic Report Generation 

The developed solution provides complete flexibility in generating dynamic reports. Users can import data from external sources (APIs) or local sources and create customized reports. 

Each column in the report is fully configurable—users can set custom names, styles, and apply complex calculation formulas that integrate data from different sources.
 

Example formula: Users can combine data from Excel files with external data using formulas like =SUM([0] + {1}), where [0] represents the first column of external data, and {1} represents the second column of the exported Excel file. 

Dynamic Chart Creation & Configuration 

Users can create dynamic charts based on the generated reports. Every element of a chart—from the X and Y axes to labels and colors—can be individually configured.
For advanced customization, users have the option to create and arrange groups of charts on the page using custom HTML code, thereby adapting the layout to their specific needs. 

Interactive Charts 

In addition to being fully customizable, the charts are also interactive. Users can click on specific datasets within the chart to access additional information related to the displayed data. This functionality allows for more detailed and dynamic data exploration, contributing to a more interactive and informative experience. 

Real-time Data Communication with NATS 

To ensure a smooth and interactive experience, the application uses NATS, a scalable messaging system, to transmit data in real-time between the server and the front end. Users benefit from automatic updates to reports and charts without needing to refresh the page manually. 


Example: In Angular, the application connects to data streams using the nats.ws package, which allows for the automatic reloading of visual components when changes occur. 

Elasticsearch for Efficient Data Search 

To handle large volumes of data and enable fast queries, we integrated Elasticsearch. This solution allows efficient searches and rapid analysis of recorded data, especially in the context of log management and auditing. 

Scheduled Tasks with Quartz.NET 

To manage and automate the periodic calculation of complex reports and the generation of temporary tables (cache), we used Quartz.NET. This framework allows scheduling and execution of recurring tasks, such as the automatic generation and caching of reports, optimizing the application's overall performance. 

Simplified Request Management with MediatR 

By using MediatR, we simplified request and response management within the application, applying the principle of decoupling between components. This solution reduced code complexity and facilitated the integration of the CQRS (Command Query Responsibility Segregation) model. 

Excel Integration with EPPlus 

To allow users to export generated reports and charts into a usable and easily shareable format, we integrated the EPPlus library. This provides the ability to generate Excel files that include both the source data and the visualized charts.  

Database Interaction with Entity Framework Core 

The application uses Entity Framework Core for managing database interactions, enabling simplified mapping of C# classes to database tables. This eliminates the need for manual SQL code writing and ensures automatic migration of changes to the database structure. 

PostgreSQL 

The application uses PostgreSQL as its relational database management system. PostgreSQL is an open-source software known for its reliability, scalability, and performance.
Main Advantages:
  • Performance and Scalability: PostgreSQL efficiently handles large volumes of data and performs excellently in large production environments.
  • Extensibility: It supports advanced extensions and data types such as JSON, making it ideal for modern and flexible applications. 

Concourse (CI/CD) 

Concourse CI is used to automate Continuous Integration and Continuous Deployment 
(CI/CD) processes. 


With every code change on GitLab, Docker images are automatically created. Before each deployment process, a backup of the database is automatically made, and SQL scripts for database migration are executed automatically. 


The automated workflow operates as follows: 
  • Automatic Database Backup: A script is executed that saves the current database 
to an external server. 
  • Running SQL Scripts: After the backup, the necessary SQL scripts for updates or migrations are automatically run. 
  • Automatic Application Deployment: Once the database is updated, the application is automatically built and deployed. 

Results

The implementation of this solution allowed the client to improve their efficiency in data management and presentation, providing users with a flexible and interactive experience. 


Among the benefits achieved are: 
  • Customizable Reports and Charts: The complete flexibility in configuring reports and charts enabled users to tailor the solution to their specific needs. 
  • Real-Time Updates: Real-time communication via NATS ensures an interactive and dynamic experience without the need for manual page refreshes. 
  • Versatile Export: Users can easily export reports and charts in various formats, including Excel and JSON, facilitating sharing and further analysis. 
  • High Performance: The use of Elasticsearch for data searching and filtering ensured rapid response times, even with large volumes of data. 
  • Efficient Automation: Quartz.NET allowed for the automatic execution of recurring tasks, optimizing system resources and enhancing user experience. 

User Workflow 

Data ImportUsers upload data from the local database or access external data sources. 
Report Creation Dynamic reports are configured by selecting and naming columns, applying calculation formulas, and styling the columns.
Chart GenerationBased on the reports, users create customized graphical visualizations, which they can visually adjust according to their needs.
Export and InteractionReports and charts can be exported in various formats, and the charts are interactive, allowing access to additional data through clicks.