Our project accounting team imports invoices from an external billing system via SFTP drop folder. Files are XML format and land in the D365 import staging area successfully. However, about 30% of invoice files get stuck in ‘Processing’ status and never complete.
The SFTP import process picks up the files correctly, and we can see them in the Data Management workspace. But they just sit there in Processing state for hours, sometimes days. No error messages appear in the execution log. We’ve tried reprocessing stuck files manually with no success.
The file format follows our agreed XML schema, and the files that DO process successfully look identical in structure to the ones that get stuck. Has anyone experienced invoice imports hanging in processing status? We need these invoices posted for month-end close.
Check the batch job execution. SFTP imports in D365 rely on recurring batch jobs to process files from staging. If your batch server is overloaded or the specific job has errors, files will queue indefinitely. Go to System Administration > Inquiries > Batch jobs and filter for your import job name. Look at the execution history for failures.
This sounds like a data entity validation issue. D365 validates imported data against entity business logic. If validation fails but doesn’t throw a clear error, the record stays in Processing limbo. Check if your XML has any special characters, date format mismatches, or reference fields pointing to non-existent records. Even one invalid line item can hang the entire invoice file.
I’ve dealt with this exact scenario. The issue is usually in the entity sequence dependencies. Project invoices have complex relationships - they reference projects, customers, line items, and accounting dimensions. If your XML doesn’t load entities in the correct dependency order, child records can’t find their parent references and the import hangs.
Also check your file naming convention. D365 SFTP import expects specific patterns and if multiple files have similar timestamps, they might be creating race conditions.
I’ve resolved this exact SFTP import issue multiple times for project invoices. Here’s the comprehensive solution addressing all three focus areas:
Root Cause Analysis:
Your invoice files are stuck because of a combination of processing status tracking issues, file format validation problems, and batch job configuration gaps. The 30% failure rate suggests conditional data issues, not systematic failures.
1. SFTP Import Configuration:
The SFTP drop folder integration uses the Recurring Integrations Scheduler (RIS) framework. Your issue is likely in how the processing status is tracked:
- Navigate to Data Management > Framework parameters > Recurring integrations
- Check ‘Processing timeout’ setting - default is 60 minutes, but complex invoices need longer
- Verify ‘Error action’ is set to ‘Stop processing’ not ‘Continue’ (which can leave files in limbo)
- Enable ‘Detailed execution log’ to capture validation failures
2. File Format Validation:
Your XML schema might be valid but not optimized for D365’s entity processing:
<!-- Ensure proper entity sequencing in XML: -->
<ProjectInvoiceHeader>
<InvoiceId>INV-2024-001</InvoiceId>
<ProjectId>PROJ-100</ProjectId>
<!-- Lines MUST come after header fields -->
<InvoiceLines>
<Line><ItemId>SVC-001</ItemId></Line>
</InvoiceLines>
</ProjectInvoiceHeader>
Common format issues causing Processing hangs:
- Date fields not in ISO 8601 format (use YYYY-MM-DD)
- Decimal separators (use period not comma, even in EU locales)
- Empty tags vs missing tags (D365 treats these differently)
- Line sequence numbers must be unique integers
- Customer account references must exist before invoice import
3. Processing Status Resolution:
To fix stuck files and prevent future occurrences:
Immediate Fix for Stuck Files:
- Open SQL Server Management Studio
- Query: `SELECT * FROM DMFStagingExecutionErrors WHERE ExecutionId IN (SELECT ExecutionId FROM DMFStagingLog WHERE ExecutionStatus = ‘Processing’ AND CreatedDateTime < DATEADD(hour, -2, GETUTCDATE()))
- This reveals hidden validation errors not shown in UI
- Delete stuck staging records: `DELETE FROM DMFStagingLog WHERE ExecutionId = ‘your-stuck-id’
- Re-upload the corrected XML files
Long-term Prevention:
- Add pre-validation in your external system before SFTP upload
- Implement file naming with sequence numbers: `ProjectInvoice_YYYYMMDD_HHMMSS_001.xml
- Configure batch job parallelization: System Administration > Batch jobs > Set ‘Maximum batch threads’ to 4-8
- Create custom validation logic in Data Management to catch issues before processing
- Set up alerts for files in Processing status > 30 minutes
Batch Job Optimization:
Your batch job completes but doesn’t process files because it’s hitting the default record limit:
- Edit your recurring data import job
- Increase ‘Records per task’ from default 1000 to 5000
- Set ‘Maximum number of tasks’ to match your batch server threads
- Enable ‘Run in batch’ checkbox (sometimes gets disabled accidentally)
After implementing these changes, monitor the first few imports closely. The processing status should update within 5-10 minutes for typical invoice files. If files still hang, check the DMFStagingValidationLog table for specific field-level errors.
Look at the DMFExecutionId in the staging tables. Query DMFStagingValidationLog and DMFStagingExecutionErrors tables directly in SQL for that execution ID. The UI doesn’t always show detailed validation errors, but they’re logged in the database. I bet you’ll find foreign key violations or data type mismatches that explain why processing stalls.
I checked the batch jobs and they’re running on schedule every 15 minutes. The job status shows ‘Ended’ successfully, but the files remain in Processing. It’s like the batch job completes without actually processing the staged files. Could there be a queue or locking issue?