Hi All,
I am using Netbackup appliance 5330. The backup speed suddenly decreased. We found below error in the detailed job status
5/3/2015 5:54:24 PM - Critical bptm(pid=43225) Storage Server Error: (Storage server: PureDisk:glsdpabkp001) mtstrm_close_write_channel: Fatal error occured in Multi-Threaded Agent: Close Write Channel command failed: Cr_ErrnoException: Timed out after waiting 180s for the push threads to complete (cnt=1) V-454-96
5/3/2015 5:54:24 PM - Critical bptm(pid=43225) sts_close_handle failed: 2060019 error occurred on network socket
5/3/2015 5:54:24 PM - Error bptm(pid=43225) cannot write image to disk, media close failed with status 2060019
5/3/2015 5:55:02 PM - Info bptm(pid=43225) EXITING with status 87 <----------
5/3/2015 5:55:02 PM - Info glsdpabkp001(pid=43225) StorageServer=PureDisk:glsdpabkp001; Report=PDDO Stats (multi-threaded stream used) for (glsdpabkp001): scanned: 744978 KB, CR sent: 2627 KB, CR sent over FC: 0 KB, dedup: 99.6%, cache disabled
5/3/2015 5:59:05 PM - Info bpbkar(pid=8188) done. status: 87: media close error
5/3/2015 5:59:05 PM - end writing; write time: 2:25:08
media close error(87)
This appliance is used as Target Replication from 3 other netbackup sites. This also connected to tape library as well. We logges a case with Symantec as well ( Case id - 08693217) but they didn't find anything yet. We observered last night backups started failing with 24, 87 and running very slow throughput. We ran backup for same clients from other appliance and it is running fine. So seems some issue with this appliance.
We changed TCP time stamp as well but not getting any improvement on Speed.
# cp "/etc/sysctl.conf /etc/sysctl.conf.tcptimestampsoff"
2. Then in "/etc/sysctl.conf", change the line that reads:
net.ipv4.tcp_timestamps = 0
To instead read:
net.ipv4.tcp_timestamps = 1
Save the file and execute:
# sysctl -p