Enhance Weekly Financial Report functionality and sorting

memory-bank/archive/archive-TargetUnit-Effective-Params-20250811.md

@@ -0,0 +1,76 @@
+# Enhancement Archive: Add Effective Financial Parameters to Target Units
+
+## Summary
+Implemented integration of effective financial parameters (Effective Revenue, Effective Margin, Effective Marginality) into the Weekly Financial Reports system through QuickBooks Online (QBO) integration.
+
+## Date Completed
+2025-08-11
+
+## Key Files Modified
+- `workers/main/src/activities/weeklyFinancialReports/fetchFinancialAppData.ts` - QBO integration for effective revenue fetching
+- `workers/main/src/configs/qbo.ts` - QBO configuration with effective revenue parameters
+- `workers/main/src/services/FinApp/types.ts` - added effectiveRevenue field to Project interface
+- `workers/main/src/services/WeeklyFinancialReport/WeeklyFinancialReportFormatter.ts` - formatting new metrics in reports
+- `workers/main/src/services/WeeklyFinancialReport/WeeklyFinancialReportRepository.ts` - core business logic for calculations
+- `workers/main/src/services/WeeklyFinancialReport/WeeklyFinancialReportRepository.test.ts` - extended testing
+- `workers/main/src/services/WeeklyFinancialReport/WeeklyFinancialReportSorting.test.ts` - new sorting tests
+
+## Requirements Addressed
+- Add Effective Revenue as Target Unit parameter
+- Add Effective Margin as Target Unit parameter
+- Add Effective Marginality as Target Unit parameter
+- Integrate data from QuickBooks Online for accurate calculations
+- Implement group sorting by marginality levels (High → Medium → Low)
+- Ensure display of new metrics in financial reports
+
+## Implementation Details
+**QBO Integration:**
+- Added `qboRepo.getEffectiveRevenue()` call in `fetchFinancialAppData`
+- Configured with `effectiveRevenueMonths` parameter (default 4 months)
+- Effective revenue linked to projects through `quick_books_id`
+
+**Calculations and Sorting:**
+- Implemented effectiveRevenue, effectiveMargin, effectiveMarginality calculations in `aggregateGroupData`
+- Added advanced sorting: first by marginality levels, then by effective marginality
+- `compareMarginalityLevels` method for sorting order determination (High: 3, Medium: 2, Low: 1)
+
+**Formatting:**
+- Updated `formatDetail` to display Effective Revenue, Effective Margin, Effective Marginality
+- Added explanations in footer about effective revenue calculation period
+- Support for marginality indicators in reports
+
+## Testing Performed
+- Unit tests for new `aggregateGroupData` method with effective metrics calculations
+- Comprehensive sorting tests in `WeeklyFinancialReportSorting.test.ts` (100+ lines)
+- Verification of correct sorting by marginality levels: High → Medium → Low
+- Testing secondary sorting by effective marginality within same level
+- Validation of new field formatting in reports
+
+## Lessons Learned
+- **QBO integration**: Repository pattern is effective for external services and scales well
+- **Financial calculations**: Require particularly detailed testing due to critical importance of accuracy
+- **Level 2 complexity**: Tasks with external service integration can be more complex than expected (+401 lines for Level 2)
+- **Code organization**: Proper separation of responsibilities between data, business, and presentation layers is critical
+- **Optimization**: The `compareMarginalityLevels` method can be inlined for simplification (~15 lines savings)
+
+## Related Work
+- Related to general Weekly Financial Reports system
+- Based on existing QBORepository infrastructure
+- Complements marginality system (MarginalityCalculator, MarginalityLevel)
+- PR #95: https://github.com/speedandfunction/automatization/pull/95
+
+## Notes
+**Technical Architecture:**
+- Used Repository pattern for QBO integration
+- Preserved backward compatibility when adding new fields
+- Efficient design: minimal changes in types, focused changes in business logic
+
+**Potential Improvements:**
+- Inline `compareMarginalityLevels` method
+- Extract marginality thresholds to configuration constants
+- More strict typing for financial calculations
+
+**Time Estimates:**
+- Planned: 1-2 days (Level 2)
+- Actual: 2-3 days
+- Variance reason: underestimation of QBO integration complexity and testing volume required

workers/main/src/activities/weeklyFinancialReports/fetchFinancialAppData.ts

@@ -3,6 +3,7 @@
 import { MongoPool } from '../../common/MongoPool';
 import { TargetUnit } from '../../common/types';
 import { FinAppRepository } from '../../services/FinApp';
+import { QBORepository } from '../../services/QBO';
 
 interface GetTargetUnitsResult {
   fileLink: string;
@@ -21,23 +22,33 @@
   try {
     await mongoPool.connect();
     const repo = new FinAppRepository();
+    const qboRepo = new QBORepository();
 
     const targetUnits = await readJsonFile<TargetUnit[]>(fileLink);
     const employeeIds = getUniqueIds(targetUnits, 'user_id');
     const projectIds = getUniqueIds(targetUnits, 'project_id');
 
-    const [employees, projects] = await Promise.all([
+    const [employees, projects, effectiveRevenue] = await Promise.all([
       repo.getEmployeesByRedmineIds(employeeIds),
       repo.getProjectsByRedmineIds(projectIds),
+      qboRepo.getEffectiveRevenue(),
     ]);
 
-    await writeJsonFile(filename, { employees, projects });
+    await writeJsonFile(filename, {
+      employees,
+      projects: projects.map((project) => ({
+        ...project,
+        effectiveRevenue: project.quick_books_id
+          ? effectiveRevenue[project.quick_books_id]?.totalAmount || 0
+          : 0,
+      })),
+    });
 
     return { fileLink: filename };
   } catch (err) {
     const message = err instanceof Error ? err.message : String(err);
 
     throw new AppError('Failed to get Fin App Data', message);
   } finally {
     await mongoPool.disconnect();
   }

workers/main/src/configs/qbo.ts

@@ -12,7 +12,7 @@ export const qboConfig = {
   tokenPath: '/oauth2/v1/tokens/bearer',
   tokenExpirationWindowSeconds: 300,
   effectiveRevenueMonths: parseInt(
-    process.env.QBO_EFFECTIVE_REVENUE_MONTHS || '3',
+    process.env.QBO_EFFECTIVE_REVENUE_MONTHS || '4',
   ),
 };
 

workers/main/src/services/FinApp/types.ts

@@ -21,6 +21,7 @@ export interface Project {
    */
   quick_books_id?: number;
   history?: History;
+  effectiveRevenue?: number;
   [key: string]: unknown;
 }
 

workers/main/src/services/WeeklyFinancialReport/WeeklyFinancialReportFormatter.ts

@@ -1,10 +1,12 @@
 import { formatCurrency } from '../../common/formatUtils';
+import { formatDateToISOString } from '../../common/utils';
+import { qboConfig } from '../../configs/qbo';
 import {
   HIGH_MARGINALITY_THRESHOLD,
   MEDIUM_MARGINALITY_THRESHOLD,
 } from '../../configs/weeklyFinancialReport';
 
-export interface formatSummaryInput {
+export interface FormatSummaryInput {
   reportTitle: string;
   highGroups: string[];
   mediumGroups: string[];
@@ -20,6 +22,9 @@ export interface FormatDetailInput {
   marginAmount: number;
   marginalityPercent: number;
   indicator: string;
+  effectiveRevenue: number;
+  effectiveMargin: number;
+  effectiveMarginality: number;
 }
 
 const spacer = ' '.repeat(4);
@@ -34,20 +39,26 @@ export class WeeklyFinancialReportFormatter {
     marginAmount,
     marginalityPercent,
     indicator,
+    effectiveRevenue,
+    effectiveMargin,
+    effectiveMarginality,
   }: FormatDetailInput) =>
-    `${indicator} *${groupName}* (${groupTotalHours}h)\n` +
+    `*${groupName}* (${groupTotalHours}h)\n` +
     `${spacer}*Period*: ${currentQuarter}\n` +
     `${spacer}*Revenue*: ${formatCurrency(groupTotalRevenue)}\n` +
     `${spacer}*COGS*: ${formatCurrency(groupTotalCogs)}\n` +
     `${spacer}*Margin*: ${formatCurrency(marginAmount)}\n` +
-    `${spacer}*Marginality*: ${marginalityPercent.toFixed(0)}%\n\n`;
+    `${spacer}*Marginality*: ${marginalityPercent.toFixed(0)}%\n` +
+    `${spacer}*Effective Revenue*: ${formatCurrency(effectiveRevenue)}\n` +
+    `${spacer}*Effective Margin*: ${formatCurrency(effectiveMargin)}\n` +
+    `${spacer}*Effective Marginality*: ${indicator} ${effectiveMarginality.toFixed(0)}%\n\n`;
 
   static formatSummary = ({
     reportTitle,
     highGroups,
     mediumGroups,
     lowGroups,
-  }: formatSummaryInput) => {
+  }: FormatSummaryInput) => {
     let summary = `${reportTitle}\n`;
 
     if (highGroups.length) {
@@ -74,11 +85,28 @@ export class WeeklyFinancialReportFormatter {
     return summary;
   };
 
-  static formatFooter = (totalHours: number) =>
-    `\n*Total hours*: ${totalHours}h\n\n` +
-    '*Notes:*\n' +
-    '1. *Contract Type* is not implemented\n' +
-    '2. *Effective Revenue* is not implemented\n' +
-    '3. *Dept Tech* hours are not implemented\n\n' +
-    `*Legend*: Marginality :arrowup: ≥${HIGH_MARGINALITY_THRESHOLD}%   :large_yellow_circle: ${MEDIUM_MARGINALITY_THRESHOLD}-${HIGH_MARGINALITY_THRESHOLD - 1}%  :arrowdown: <${MEDIUM_MARGINALITY_THRESHOLD}%`;
+  private static calculateDateWindow() {
+    const endDate = new Date();
+    const startDate = new Date(endDate);
+
+    startDate.setMonth(endDate.getMonth() - qboConfig.effectiveRevenueMonths);
+
+    return {
+      startDate: formatDateToISOString(startDate),
+      endDate: formatDateToISOString(endDate),
+    };
+  }
+
+  static formatFooter = (totalHours: number) => {
+    const { startDate, endDate } = this.calculateDateWindow();
+
+    return (
+      `\n*Total hours*: ${totalHours}h\n\n` +
+      '*Notes:*\n' +
+      '1. *Contract Type* is not implemented\n' +
+      `2. *Effective Revenue* calculated for the last ${qboConfig.effectiveRevenueMonths} months (${startDate} - ${endDate})\n` +
+      '3. *Dept Tech* hours are not implemented\n\n' +
+      `*Legend*: Marginality :arrowup: ≥${HIGH_MARGINALITY_THRESHOLD}%   :large_yellow_circle: ${MEDIUM_MARGINALITY_THRESHOLD}-${HIGH_MARGINALITY_THRESHOLD - 1}%  :arrowdown: <${MEDIUM_MARGINALITY_THRESHOLD}%`
+    );
+  };
 }

workers/main/src/services/WeeklyFinancialReport/WeeklyFinancialReportRepository.test.ts

@@ -2,10 +2,8 @@ import { describe, expect, it } from 'vitest';
 
 import { WeeklyFinancialReportRepository } from './WeeklyFinancialReportRepository';
 
-describe('WeeklyFinancialReportRepository', () => {
-  const repo = new WeeklyFinancialReportRepository();
-
-  const targetUnits = [
+const createBasicTestData = () => ({
+  targetUnits: [
     {
       group_id: 1,
       group_name: 'Group A',
@@ -56,48 +54,95 @@ describe('WeeklyFinancialReportRepository', () => {
       spent_on: '2024-06-01',
       total_hours: 10,
     },
-  ];
-  const employees = [
+  ],
+  employees: [
     { redmine_id: 100, history: { rate: { '2024-01-01': 100 } } },
     { redmine_id: 101, history: { rate: { '2024-01-01': 200 } } },
     { redmine_id: 102, history: { rate: { '2024-01-01': 300 } } },
     { redmine_id: 103, history: { rate: { '2024-01-01': 900 } } },
     { redmine_id: 104, history: { rate: { '2024-01-01': 700 } } },
-  ];
-  const projects = [
+  ],
+  projects: [
     { redmine_id: 10, history: { rate: { '2024-01-01': 500 } } },
     { redmine_id: 20, history: { rate: { '2024-01-01': 1000 } } },
     { redmine_id: 30, history: { rate: { '2024-01-01': 1500 } } },
     { redmine_id: 40, history: { rate: { '2024-01-01': 1300 } } },
-  ];
+  ],
+});
+
+const createMarginalityTestData = () => ({
+  targetUnits: [
+    {
+      group_id: 1,
+      group_name: 'Group A',
+      project_id: 10,
+      project_name: 'Project X',
+      user_id: 100,
+      username: 'Alice',
+      spent_on: '2024-06-01',
+      total_hours: 10,
+    },
+    {
+      group_id: 2,
+      group_name: 'Group B',
+      project_id: 20,
+      project_name: 'Project Y',
+      user_id: 101,
+      username: 'Bob',
+      spent_on: '2024-06-01',
+      total_hours: 10,
+    },
+    {
+      group_id: 3,
+      group_name: 'Group C',
+      project_id: 30,
+      project_name: 'Project Z',
+      user_id: 102,
+      username: 'Charlie',
+      spent_on: '2024-06-01',
+      total_hours: 10,
+    },
+  ],
+  employees: [
+    { redmine_id: 100, history: { rate: { '2024-01-01': 50 } } },
+    { redmine_id: 101, history: { rate: { '2024-01-01': 50 } } },
+    { redmine_id: 102, history: { rate: { '2024-01-01': 50 } } },
+  ],
+  projects: [
+    { redmine_id: 10, history: { rate: { '2024-01-01': 100 } } }, // 50% marginality
+    { redmine_id: 20, history: { rate: { '2024-01-01': 200 } } }, // 75% marginality
+    { redmine_id: 30, history: { rate: { '2024-01-01': 150 } } }, // 67% marginality
+  ],
+});
+
+describe('WeeklyFinancialReportRepository', () => {
+  const repo = new WeeklyFinancialReportRepository();
 
   it('generates a report with summary and details', async () => {
+    const testData = createBasicTestData();
     const { summary, details } = await repo.generateReport({
-      targetUnits,
-      employees,
-      projects,
+      targetUnits: testData.targetUnits,
+      employees: testData.employees,
+      projects: testData.projects,
     });
 
     expect(typeof summary).toBe('string');
     expect(typeof details).toBe('string');
     expect(summary.length).toBeGreaterThan(0);
     expect(details.length).toBeGreaterThan(0);
 
-    // Check summary content
     expect(summary).toContain('Weekly Financial Summary for Target Units');
     expect(summary).toContain('Marginality is 55% or higher');
     expect(summary).toContain('Marginality is between 45-55%');
     expect(summary).toContain('Marginality is under 45%');
     expect(summary).toContain(
       'The specific figures will be available in the thread',
     );
-    // Group names should appear in summary
     expect(summary).toContain('Group A');
     expect(summary).toContain('Group B');
     expect(summary).toContain('Group C');
     expect(summary).toContain('Group D');
 
-    // Check details content
     expect(details).toContain('Total hours');
     expect(details).toContain('Group A');
     expect(details).toContain('Group B');
@@ -133,4 +178,24 @@ describe('WeeklyFinancialReportRepository', () => {
     expect(details).toContain('Legend');
     expect(summary).toContain('Weekly Financial Summary for Target Units');
   });
+
+  it('sorts groups by effectiveMarginality in descending order', async () => {
+    const testData = createMarginalityTestData();
+    const { details } = await repo.generateReport({
+      targetUnits: testData.targetUnits,
+      employees: testData.employees,
+      projects: testData.projects,
+    });
+
+    // Check that groups are displayed in the correct order (by descending effectiveMarginality)
+    const groupBIndex = details.indexOf('Group B');
+    const groupCIndex = details.indexOf('Group C');
+    const groupAIndex = details.indexOf('Group A');
+
+    // Group B should be first (75% marginality)
+    // Group C should be second (67% marginality)
+    // Group A should be third (50% marginality)
+    expect(groupBIndex).toBeLessThan(groupCIndex);
+    expect(groupCIndex).toBeLessThan(groupAIndex);
+  });
 });