Creating Project Status Reports

Dear #DataEngineers, No matter how confident you are in your SQL queries or ETL pipelines, never assume data correctness without validation. ETL is more than just moving data—it’s about ensuring accuracy, completeness, and reliability. That’s why validation should be a mandatory step, making it ETLV (Extract, Transform, Load & Validate). Here are 20 essential data validation checks every data engineer should implement (not all pipeline require all of these, but should follow a checklist like this): 1. Record Count Match – Ensure the number of records in the source and target are the same. 2. Duplicate Check – Identify and remove unintended duplicate records. 3. Null Value Check – Ensure key fields are not missing values, even if counts match. 4. Mandatory Field Validation – Confirm required columns have valid entries. 5. Data Type Consistency – Prevent type mismatches across different systems. 6. Transformation Accuracy – Validate that applied transformations produce expected results. 7. Business Rule Compliance – Ensure data meets predefined business logic and constraints. 8. Aggregate Verification – Validate sum, average, and other computed metrics. 9. Data Truncation & Rounding – Ensure no data is lost due to incorrect truncation or rounding. 10. Encoding Consistency – Prevent issues caused by different character encodings. 11. Schema Drift Detection – Identify unexpected changes in column structure or data types. 12. Referential Integrity Checks – Ensure foreign keys match primary keys across tables. 13. Threshold-Based Anomaly Detection – Flag unexpected spikes or drops in data volume or values. 14. Latency & Freshness Validation – Confirm that data is arriving on time and isn’t stale. 15. Audit Trail & Lineage Tracking – Maintain logs to track data transformations for traceability. 16. Outlier & Distribution Analysis – Identify values that deviate from expected statistical patterns. 17. Historical Trend Comparison – Compare new data against past trends to catch anomalies. 18. Metadata Validation – Ensure timestamps, IDs, and source tags are correct and complete. 19. Error Logging & Handling – Capture and analyze failed records instead of silently dropping them. 20. Performance Validation – Ensure queries and transformations are optimized to prevent bottlenecks. Data validation isn’t just a step—it’s what makes your data trustworthy. What other checks do you use? Drop them in the comments! #ETL #DataEngineering #SQL #DataValidation #BigData #DataQuality #DataGovernance

10 Reporting Tips I have sent 100s of reports. And overtime I have found what works and what doesn't work. Here are my top 10 tips: 1. Audience Identify Key Stakeholders: Determine the specific individuals or departments who will benefit most from the report. Customize Content: Tailor the report’s content to address the unique needs or interests of different audience segments. Feedback Loop: Regularly solicit feedback from the audience to continuously improve the relevance and effectiveness of the report. 2. Timing Align with Business Cycles: Schedule reports in sync with business cycles, like quarterly financial periods. Anticipate Needs: Proactively adjust the reporting frequency during critical business phases. Automate Reminders: Use scheduling tools to automate the distribution process and ensure timely delivery. 3. Business Data Integrate KPIs: Include key performance indicators relevant to the business operations. Dynamic Data Sources: Use real-time data feeds to enhance the report’s immediacy and relevance. Contextual Analysis: Provide analytical insights, comparing operational data trends over time or against industry benchmarks. 4. Declutter Prioritize Data: Focus on the most critical data points that drive decision-making. Visual Simplicity: Use clean, simple visuals to enhance readability and comprehension. Minimalist Design: Adopt a minimalist design approach to reduce cognitive overload. 5. Reusable Template Design: Develop templates that ensure consistency and ease of adaptation for presentations. Modular Sections: Create the report in modular sections for easy extraction and reuse. Adaptable Formats: Ensure the report can be easily converted into different formats without losing its essence. 6. Format Interactivity in Digital Formats: Utilize interactive elements in digital formats like Excel or web-based reports. Print-Friendly Options: Offer a print-friendly version for those who prefer physical copies. 7. Push vs Pull Automated Alerts: Set up automated alerts for new report availability in pull systems. Customizable Push Options: Allow recipients to customize the frequency and type of reports they receive. Secure Access: Ensure secure, easy access for pull systems, particularly for sensitive financial data. 8. Comments Executive Summaries: Include an executive summary highlighting key insights and decisions. Actionable Recommendations: Offer clear, actionable recommendations based on the report’s findings. 9. Standard Brand Alignment: Ensure the report’s visual elements align with the company’s branding guidelines. 10. Self-Explanatory Infographics: Use infographics to make complex data more understandable. Layered Information: Present information in layers, with summaries leading to detailed analysis. Guided Navigation: Include a table of contents or navigation aids to guide the reader through the report. 👉 What is your best reporting tips?

Did you know Power BI report metadata can contain data points? In certain circumstances, it can contain column values. This means report metadata can contain sensitive info and should be treated appropriately. This is by design in how visuals save their config. One example is when you have a matrix visual with a field in the "Columns" well and disable "Auto-size column widths". To save column widths, the visual config uses the name of each column - which is a value. These values could be OII or PII - names, emails, and so forth. This also means that if someone has access to a .pbix, .pbip, or .pbit file, unless it's a .pbix with a sensitivity label, they can theoretically access this metadata and open it to view sensitive information. Even if they don't have access to the underlying model or data sources. Many people don't know this, so I share it here. This is relevant in circumstances like the following: - When and where you save report files. - Tools that ingest report metadata. - AI / Copilot ingestion of report metadata. In the last months I've seen many more people using report metadata. I've heard multiple people say things like "it's just metadata, it contains no data points". This is not true! Don't make assumptions! Especially when it comes to other tools and AI. #PowerBI #MicrosoftFabric #DataPrivacy #DataSecurity #Godot

Turn One Messy Report into Multiple Powerful Views Your Chart of Accounts tells you WHAT happened. But it can't tell you WHERE it happened or WHO spent the money. That's the problem with standard P&L reports. You see $50,000 in advertising spend. But you have no idea if that was Sales, Marketing, or IT. You see $200,000 in salaries. But which team consumed that? Engineering? Operations? Without classes, everything is just mixed together. → How Classes Work: The 2 Dimension System Classes add a second dimension to your reporting. Dimension 1 is WHAT (the GL account). Dimension 2 is WHERE (the class tag). So now when you categorize that $50,000 advertising expense, you're also tagging it... $30k to Sales, $20k to Marketing. That $200,000 in salaries? $120k to Engineering, $80k to Operations. Crystal clear accountability. → Two Ways to Track Departmental Spending by Class Once you start using classes, you have two main reporting options. The first is Summary P&L by Class. This gives you a clean, simple view of all departments at once. Perfect for comparing spending across classes. No filters needed, everything appears automatically. The second is Summary P&L with Class Filter. This keeps your P&L structure clean (same accounts as standard P&L). Want Sales only? Click filter. Done. Want Engineering only? Click filter. Done. Both formats have their place depending on what you're trying to analyze. → What You Can Track with Classes Classes aren't just for departments. You can track by: - Departments (Sales, HR, IT) - Product Lines (Software, Hardware) - Locations (NYC, LA, Austin) - Projects (Building A, B, C) The flexibility is endless. → Why Classes Are a Total Advantage Classes answer the real questions. Which teams drive results? You get multiple views from the same data. Slice by team, project, or location. True accountability kicks in. Every dollar traces back to who spent it. And it scales beautifully. From 10 to 100+ employees without breaking a sweat. The challenge? Setting up class reporting manually in QuickBooks takes forever. You're exporting data, building pivot tables, creating filters... At Model Wiz, you connect your QuickBooks Online account and we automatically generate these dashboards for you. All the views, all the filters, ready to go. Are you using classes in your business? Let me know in the comments 👇

July's Power BI update has one feature worth your attention. Everything else can wait. 𝐑𝐞𝐩𝐨𝐫𝐭-𝐰𝐢𝐝𝐞 𝐭𝐡𝐞𝐦𝐢𝐧𝐠, straight from the 𝐅𝐨𝐫𝐦𝐚𝐭 𝐩𝐚𝐧𝐞. Customise current theme now sits in the Format pane. Click an empty part of the canvas, and it's there. → Change report-wide colours with a palette, without losing your other customisations → Set defaults for borders, backgrounds, shadows, header icons, corner radius and padding → Apply page size, background and wallpaper across every page at once → Push filter pane formatting to current and new pages → Export the result as a theme and reuse it or share it with your team Every new visual you drop on the canvas inherits it. The part I care about: consistency stops being a cleanup job at the end of the build. It becomes the default. Individual page and visual styling still wins where you need it, and you can reset any setting back to the theme. Still in preview. Turn on Modern visual defaults and customise theme improvements in Options first. #PowerBI #MicrosoftFabric #DataViz #BusinessIntelligence

Canva for Football Scout Reports 🎨⚽ One of the biggest benefits of building my own platform is having a playground to experiment, learn, and push boundaries without constraints. I'm proud of the player profiles I've developed - but I kept hearing the same feedback: "Can I customise this for my own reports?" So I built it. The Scout Report Builder pulls player data directly into a drag-and-drop canvas where you can:  → Add radar charts, scatter plots, stats tables, recent form  → Full customisation on every element - colours, sizes, positioning  → Include your own branding and commentary  → Export high-quality images ready for presentations The idea is simple: data-driven visuals to support your analysis, with your own perspective and insights layered on top. No code. No design skills. Just click, customise, drag and drop. This was an ambitious feature to land with a huge ceiling for where it can go - so I'd genuinely appreciate any feedback from scouts, analysts, or anyone who creates player reports. What would make this useful for your workflow?

The first thing an analyst should build isn't a dashboard. It's trust in the data. Many people think an analyst's job starts with Power BI, Tableau, SQL, Python, or whatever reporting tool they use. I think it starts much earlier. Before I build a dashboard, define KPIs, or create reports, I spend time understanding the data behind it. I ask questions like: 📊 Do the total record counts make sense? 📁 Do the subcategory counts also make sense? (I've seen many cases where the grand total was correct, but records were assigned to the wrong categories. The dashboard looked right, but the business story was wrong.) 🔍 Are there duplicate or missing records? 🔄 Do values reconcile across different source systems? 📐 Are business rules reflected correctly in the data? ⏰ Did the ETL or scheduled refresh complete successfully? How fresh is the data? 💡 Most importantly... does the data actually make business sense? One lesson I've learned throughout my career is that the most dangerous data issues are often the ones hiding in plain sight. Sometimes the total count is perfect. But... ⚠️ One product line is overstated. ⚠️ One customer segment is understated. ⚠️ One business unit is mapped incorrectly. Everything appears reasonable—until someone makes an important business decision based on it. Some of the biggest contributions I've made weren't building dashboards. They were identifying data quality issues before executives ever saw a report. Fixing problems at the source prevented inaccurate KPIs, misleading dashboards, and costly downstream remediation. 📌 A dashboard doesn't create trust. Trustworthy data creates trustworthy dashboards. As analysts, one of our most important responsibilities isn't simply to visualize data—it's to ensure the information behind every KPI, report, and dashboard is: ✅ Accurate ✅ Complete ✅ Consistent ✅ Timely ✅ Aligned with the business Because at the end of the day... 🗑️ Garbage in. Garbage out. Or, as I like to say: 📈 Great dashboards don't start with visualization. They start with data you can trust. #DataAnalytics #BusinessIntelligence #DataQuality #DataGovernance #SQL #PowerBI #Tableau #Analytics #DataEngineering

𝗕𝗮𝗱 𝗱𝗮𝘁𝗮 𝗰𝗿𝗲𝗮𝘁𝗲𝘀 𝗯𝗮𝗱 𝗱𝗲𝗰𝗶𝘀𝗶𝗼𝗻𝘀. That is why data validation is one of the most important skills every analyst should understand. Before analysis, dashboards, reports, or machine learning begin, the dataset must be checked for accuracy, completeness, structure, and consistency. Because even a small issue in the data can lead to wrong insights later. 𝗛𝗲𝗿𝗲 𝗮𝗿𝗲 𝟲 𝗱𝗮𝘁𝗮 𝘃𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻 𝘁𝗲𝗰𝗵𝗻𝗶𝗾𝘂𝗲𝘀 𝗲𝘃𝗲𝗿𝘆 𝗮𝗻𝗮𝗹𝘆𝘀𝘁 𝘀𝗵𝗼𝘂𝗹𝗱 𝗸𝗻𝗼𝘄: → 𝗗𝗮𝘁𝗮 𝗧𝘆𝗽𝗲 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻 Ensures columns store values in the correct format before analysis begins. → 𝗥𝗮𝗻𝗴𝗲 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻 Checks whether values fall within expected business, logical, or operational limits. → 𝗠𝗮𝗻𝗱𝗮𝘁𝗼𝗿𝘆 𝗙𝗶𝗲𝗹𝗱 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻 Confirms important fields are not blank, missing, or incomplete. → 𝗗𝘂𝗽𝗹𝗶𝗰𝗮𝘁𝗲 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻 Detects repeated records that can inflate counts, revenue, users, or transactions. → 𝗙𝗼𝗿𝗺𝗮𝘁 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻 Verifies emails, dates, phone numbers, IDs, and codes follow required patterns. → 𝗖𝗼𝗻𝘀𝗶𝘀𝘁𝗲𝗻𝗰𝘆 𝗩𝗮𝗹𝗶𝗱𝗮𝘁𝗶𝗼𝗻 Ensures related fields do not contradict each other across rows or systems. 𝗧𝗵𝗲 𝗿𝗲𝗮𝗹 𝗹𝗲𝘀𝘀𝗼𝗻: Data validation is not just a cleaning step. It is the quality gate between raw data and reliable analysis. If the data type is wrong, the calculation breaks. If the range is wrong, outliers mislead. If fields are missing, reports become incomplete. If duplicates exist, numbers get inflated. If formats vary, systems fail to match records. If consistency is missing, trust disappears. A good analyst does not only analyze data. A good analyst first makes sure the data can be trusted. Save this if you are learning data analytics, SQL, Python, reporting, or preparing for analyst interviews.

Data quality is fundamental to achieving reliable, impactful program outcomes, especially within the complex landscape of humanitarian and public health interventions. This document, Data Quality and Quality Improvement Training by USAID, provides an in-depth approach to data quality assessment, introducing critical tools like the Data Quality Assessment (DQA) and Routine Data Quality Assessment (RDQA) frameworks. These tools are designed to help organizations evaluate, maintain, and enhance the accuracy, consistency, and timeliness of their data, empowering them to make informed, data-driven decisions. This guide is essential for M&E professionals and program managers who are responsible for data integrity across service sites and reporting systems. It outlines step-by-step processes for verifying data at multiple levels, from on-site service data checks to system-wide evaluations, ensuring that data collection and reporting are aligned with high standards of quality. Practical tools, including Excel-based dashboards and real-time monitoring checklists, support these assessments, allowing for immediate insights into areas that need improvement. Beyond verification, the document emphasizes the value of building data quality into everyday processes, from staff training to cross-referencing data sources, and includes strategies for continuous quality improvement. This resource is indispensable for anyone committed to enhancing program accountability, data reliability, and ultimately, the effectiveness of humanitarian interventions.

#LookerStudio - New available feature The 🎨modern charts🎨 are now available in Preview for all the users of Looker Studio.🎊 This refreshment of charts is at the same time a graphical revamping and additional options/capabilities for most of the available chart types: Time series, Bar charts, Pie/donut charts, Scatter/bubble charts, and Line/Combo charts I've already tested them privately in November and provided my feedback to help the Looker Studio team. And some of them have been implemented 😊 Thank you Edgar Aguillon, Sean Zinsmeister and all the team. If you want to test them too, it is super easy: Go to the "Theme & Layout" panel and toggle on the "Modern charts" option. It is a preview, so prefer testing on report drafts or replications. Meanwhile, here an overview of significant additions. ➡️ Chart types with axis For all the chart types with axis, you can now decide to hide separately, for X or Y axis, the title, the interval metric label or/and the line. I've asked for it and I'm very happy to see it implemented. It looks a small thing, but for a daily usage of Looker Studio, trying to maximize everytime the data pixel ratio, it is a game changer for me. No more rectangle with the same color than the background to hide specific axis elements. 🤩 You will see below how I took advantage differently of this option for some of the displayed charts. ➡️ Time Series - A fresh gradient effect - Separate and super customizable options to hide axis elements: Here, I remove completely the Y-axis because we already displayed the label of each data point ➡️ Bar Charts - Corner radius - Separate and super customizable options to hide axis elements: Here, hiding the X-axis...again, data labels are displayed for each bar. ➡️ Donut & Pie Charts - Border Radius Yes, even the pie charts have been pampered with these fresh rounded corners. ➡️ Scatter & Bubble Charts - Bubble opacity - Separate and super customizable options to hide axis elements: Here in my bubble chart, for a confirmation of metrics correlation, I don't need the axis lines and labels but just be sure of the metrics. The grid is enough. Voila for the review and the nice additions. And now, let's ask more 😈: I would also love to see the following improvements about LS charts taken in account: - Solve the current problems around pivot tables raised on Google Cloud community - Time series: Fix the problem related to "Range by data" (system error that force to use line charts instead of time series) - Modern Bar charts: Get an option to control the corner radius of bars - Donut charts: An option to customize the number of decimals for percentage labels - Donut charts: An option to display the labels outside and around the pie, close to each sector (excel-like) - Scatter/bubble chars: Be able to draw more bubbles and scatter points and solve the problem of slowness on pages with these kind of charts Do you agree with me? Do you have any other ideas to share?