We have an issue with big application download through BITS to DWAN clients. It contain single file about 350MB in size. About 10% of clients end up with error 0x800705b4 Failed to locate content. Other applications with smaller download sizes show about 1% of such errors.
Boundaries set by AD Sites. DPs is assigned to boundary groups.
All distribution points contain needed content and it accessible through HTTP.
We have strict BITS policy that allow only ~200kbps download per client to prevent DWAN high load.
We have successes and errors from same client physical LAN locations without visible reason. Download starts over and over and always fail on same clients.
Updating content on DP doesn`t help to solve problem. Revision change visible on clients, download result stays same - error.
At this point we can`t make local DPs in clients proximity because of star network topology with little clients in each location.
DataTransferService.log from client:
UpdateURLWithTransportSettings(): OLD URL - http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
UpdateURLWithTransportSettings(): NEW URL - http://rsb-expmossccm4.domain:80/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
DTSJob {A39EB537-E09D-4982-9C05-3CA7304B34BB} created to download from 'http://rsb-expmossccm4.domain:80/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1' to 'C:\WINDOWS\ccmcache\i'.
DTSJob {A39EB537-E09D-4982-9C05-3CA7304B34BB} in state 'DownloadingManifest'.
Raising event:
instance of CCM_CcmHttp_Status
{
ClientID = "GUID:CA401F42-3039-4301-B44E-3BA409E2DEBD";
DateTime = "20160701010128.468000+000";
HostName = "rsb-expmossccm4.domain";
HRESULT = "0x00000000";
ProcessID = 2556;
StatusCode = 0;
ThreadID = 5168;
};
UpdateURLWithTransportSettings(): OLD URL - http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/sccm?/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1/
UpdateURLWithTransportSettings(): NEW URL - http://rsb-expmossccm4.domain:80/SMS_DP_SMSPKG$/sccm?/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1/
UpdateURLWithTransportSettings(): OLD URL - http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1/sccm?/setup.exe
UpdateURLWithTransportSettings(): NEW URL - http://rsb-expmossccm4.domain:80/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1/sccm?/setup.exe
DTSJob {A39EB537-E09D-4982-9C05-3CA7304B34BB} in state 'RetrievedManifest'.
DTSJob {A39EB537-E09D-4982-9C05-3CA7304B34BB} in state 'PendingDownload'.
DTSFlag is 0x0000010e
Exclude file list:
Not using branch cache option
DTSJob {A39EB537-E09D-4982-9C05-3CA7304B34BB} in state 'DownloadingData'.
CDTSJob::HandleErrors: DTS Job '{A39EB537-E09D-4982-9C05-3CA7304B34BB}' BITS Job '{043DBBC7-47E7-4235-BC13-3D3FE1211680}' under user 'S-1-5-18' OldErrorCount 0 NewErrorCount 1 ErrorCode 0x80072EE2
CDTSJob::HandleErrors: DTS Job ID='{A39EB537-E09D-4982-9C05-3CA7304B34BB}' URL='http://rsb-expmossccm4.domain:80/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1' ProtType=1
DTSJob {A39EB537-E09D-4982-9C05-3CA7304B34BB} in state 'Cancelled'.
DTSJob {A39EB537-E09D-4982-9C05-3CA7304B34BB} cancelled by client.
ContentTransferManager.log:
Starting CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC}.
Created CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC} for user S-1-5-18
Created and Sent Location Request '{0F238B6B-BD2F-443B-BF88-7DFD2ADA1365}' for package Content_da778e78-d42e-41a7-a8e3-26db4489e8a3
CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC} entered phase CCM_DOWNLOADSTATUS_DOWNLOADING_DATA
Queued location request '{0F238B6B-BD2F-443B-BF88-7DFD2ADA1365}' for CTM job '{6F5A0372-255B-4A4B-9813-BC83372570CC}'.
Persisted locations for CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC}:
(LOCAL) http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
(LOCAL) http://rsb-expmossccm5.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
(LOCAL) http://RSB-EXMOSSCCM1.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Failed get the value of Content_b54f5c1b-c3c0-4458-827e-e19a03ca9eb2.1
Failed get the value of Content_5e35a542-da31-4aab-912b-4591f709582d.1
Failed get the value of Content_4e7eec7e-f33c-4a3f-a06b-5ddf8f810d55.1
Failed get the value of Content_f47f6a1d-dc45-409c-9bf1-2e3b2046d0f3.1
Failed get the value of Content_ca511e1f-79c5-486a-8bd1-653e93106835.1
CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC} (corresponding DTS job {A39EB537-E09D-4982-9C05-3CA7304B34BB}) started download from 'http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1' for full content download.
CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC} entered phase CCM_DOWNLOADSTATUS_DOWNLOADING_DATA
CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC} entered phase CCM_DOWNLOADSTATUS_DOWNLOADING_DATA
CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC} entered phase CCM_DOWNLOADSTATUS_DOWNLOADING_DATA
CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC} entered phase CCM_DOWNLOADSTATUS_DOWNLOADING_DATA
Persisted locations for CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC}:
(LOCAL) http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
(LOCAL) http://RSB-EXMOSSCCM1.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
(LOCAL) http://rsb-expmossccm5.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
CAS.log from client:
**** Received request for content Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1, size(KB) 322848, under context System with priority Medium.
CacheManager: There are currently 3000320 bytes used for cached content items (5 total, 1 active, 4 tombstoned, 0 expired).
----- CacheManager: Grooming cache, target free space size is 330596352 bytes, allow deletion of active items = 'FALSE', allow deletion of pinned items = 'FALSE'.
CacheManager: There are currently 3000320 bytes used for cached content items (5 total, 1 active, 4 tombstoned, 0 expired).
----- CacheManager: Goal free space size for groom operation is 330596352 bytes, cache already has 5239879680 bytes available, so no grooming is required at this time.
User policy requested with no user credentials.
Invalid user.
Submitted CTM job {6F5A0372-255B-4A4B-9813-BC83372570CC} to download Content Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1 under context System
Successfully created download request {DFE12BE7-4849-4CE3-89E8-EFA535934A8E} for content Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Location update from CTM for content Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1 and request {DFE12BE7-4849-4CE3-89E8-EFA535934A8E}
Download location found 0 - http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Download location found 1 - http://rsb-expmossccm5.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Download location found 2 - http://RSB-EXMOSSCCM1.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Download location found 3 - http://RSB-EXMOSSCCM.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Download request only, ignoring location update
Download started for content Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Location update from CTM for content Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1 and request {DFE12BE7-4849-4CE3-89E8-EFA535934A8E}
Download location found 0 - http://RSB-EXMOSSCCM1.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Download location found 1 - http://rsb-expmossccm5.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Download location found 2 - http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Download location found 3 - http://RSB-EXMOSSCCM.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1
Download request only, ignoring location update
Download failed for content Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1 under context System, error 0x800705b4
Download failed for download request {DFE12BE7-4849-4CE3-89E8-EFA535934A8E}
Raising event:
[SMS_CodePage(866), SMS_LocaleID(1049)]
instance of SoftDistDownloadFailedEvent
{
ClientID = "GUID:CA401F42-3039-4301-B44E-3BA409E2DEBD";
DateTime = "20160701020122.988000+000";
MachineName = "DT-KYA000724180";
PackageId = "Content_da778e78-d42e-41a7-a8e3-26db4489e8a3";
PackageName = "Content_da778e78-d42e-41a7-a8e3-26db4489e8a3";
PackageVersion = "1";
ProcessID = 2556;
SiteCode = "MOS";
ThreadID = 5672;
};
Successfully raised Download Failed event.
Error: DeleteDirectory:- Failed to delete Directory C:\WINDOWS\ccmcache\i.BCWork with Error 0x00000002.
Releasing content request {DFE12BE7-4849-4CE3-89E8-EFA535934A8E}
LocationServices.log
Current AD site of machine is Rsb-2KYA3
Current AD site of machine is Rsb-2KYA3
sRelatedContentIDs is <RelatedContentIDs><RelatedContentID ID="Content_b54f5c1b-c3c0-4458-827e-e19a03ca9eb2.1"/><RelatedContentID ID="Content_5e35a542-da31-4aab-912b-4591f709582d.1"/><RelatedContentID ID="Content_4e7eec7e-f33c-4a3f-a06b-5ddf8f810d55.1"/><RelatedContentID ID="Content_f47f6a1d-dc45-409c-9bf1-2e3b2046d0f3.1"/><RelatedContentID ID="Content_ca511e1f-79c5-486a-8bd1-653e93106835.1"/></RelatedContentIDs>. Length = 394.
Calling back with the following distribution points
Distribution Point='http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1', Locality='LOCAL', DPType='SERVER', Version='7958', Capabilities='<Capabilities SchemaVersion="1.0"><Property Name="SSLState" Value="0"/></Capabilities>', Signature='http://rsb-expmossccm4.domain/SMS_DP_SMSSIG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1.tar', ForestTrust='TRUE',
Distribution Point='http://rsb-expmossccm5.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1', Locality='LOCAL', DPType='SERVER', Version='7958', Capabilities='<Capabilities SchemaVersion="1.0"><Property Name="SSLState" Value="0"/></Capabilities>', Signature='http://rsb-expmossccm5.domain/SMS_DP_SMSSIG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1.tar', ForestTrust='TRUE',
Distribution Point='http://RSB-EXMOSSCCM1.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1', Locality='LOCAL', DPType='SERVER', Version='7958', Capabilities='<Capabilities SchemaVersion="1.0"><Property Name="SSLState" Value="0"/></Capabilities>', Signature='http://RSB-EXMOSSCCM1.domain/SMS_DP_SMSSIG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1.tar', ForestTrust='TRUE',
Distribution Point='http://RSB-EXMOSSCCM.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1', Locality='REMOTE', DPType='SERVER', Version='7958', Capabilities='<Capabilities SchemaVersion="1.0"><Property Name="SSLState" Value="0"/></Capabilities>', Signature='http://RSB-EXMOSSCCM.domain/SMS_DP_SMSSIG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1.tar', ForestTrust='TRUE',
Calling back with locations for location request {0F238B6B-BD2F-443B-BF88-7DFD2ADA1365}
Executing Task LSRefreshLocationsTask
Current AD site of machine is Rsb-2KYA3
Current AD site of machine is Rsb-2KYA3
sRelatedContentIDs is <RelatedContentIDs><RelatedContentID ID="Content_b54f5c1b-c3c0-4458-827e-e19a03ca9eb2.1"/><RelatedContentID ID="Content_5e35a542-da31-4aab-912b-4591f709582d.1"/><RelatedContentID ID="Content_4e7eec7e-f33c-4a3f-a06b-5ddf8f810d55.1"/><RelatedContentID ID="Content_f47f6a1d-dc45-409c-9bf1-2e3b2046d0f3.1"/><RelatedContentID ID="Content_ca511e1f-79c5-486a-8bd1-653e93106835.1"/></RelatedContentIDs>. Length = 394.
Calling back with the following distribution points
Distribution Point='http://RSB-EXMOSSCCM1.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1', Locality='LOCAL', DPType='SERVER', Version='7958', Capabilities='<Capabilities SchemaVersion="1.0"><Property Name="SSLState" Value="0"/></Capabilities>', Signature='http://RSB-EXMOSSCCM1.domain/SMS_DP_SMSSIG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1.tar', ForestTrust='TRUE',
Distribution Point='http://rsb-expmossccm5.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1', Locality='LOCAL', DPType='SERVER', Version='7958', Capabilities='<Capabilities SchemaVersion="1.0"><Property Name="SSLState" Value="0"/></Capabilities>', Signature='http://rsb-expmossccm5.domain/SMS_DP_SMSSIG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1.tar', ForestTrust='TRUE',
Distribution Point='http://rsb-expmossccm4.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1', Locality='LOCAL', DPType='SERVER', Version='7958', Capabilities='<Capabilities SchemaVersion="1.0"><Property Name="SSLState" Value="0"/></Capabilities>', Signature='http://rsb-expmossccm4.domain/SMS_DP_SMSSIG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1.tar', ForestTrust='TRUE',
Distribution Point='http://RSB-EXMOSSCCM.domain/SMS_DP_SMSPKG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1', Locality='REMOTE', DPType='SERVER', Version='7958', Capabilities='<Capabilities SchemaVersion="1.0"><Property Name="SSLState" Value="0"/></Capabilities>', Signature='http://RSB-EXMOSSCCM.domain/SMS_DP_SMSSIG$/Content_da778e78-d42e-41a7-a8e3-26db4489e8a3.1.tar', ForestTrust='TRUE',
Calling back with locations for location request {0F238B6B-BD2F-443B-BF88-7DFD2ADA1365}