You can open the diagnostic dialog from the Maintenance menu in Device management. Various tools are available to monitor and diagnose the CacheDrive device.
Diagnostics Items
Disk S.M.A.R.T
Disk S.M.A.R.T (Self-Monitoring, Analysis and Reporting Technology) is a standard diagnostic tool for disk storage (HDD and SSD). Some of the parameters are commonly used, but others may be vendor specific. Also, SATA and NVMe drives are shown in different formats.
One of the important metrics is the parameter
5 Reallocation Sector Count
which indicates the number of bad sectors remapped to new spare sectors. If this counter continues to increase over time, it means the disk is deteriorating and you may want to consider replacing the CacheDrive.
For SSD, 232 Endurance Remaining is a key item, and when it decreases to below 20%, it is a sign of almost reaching to end of life.
Below is an example output of Bad Disk. Bad Sectors are increased to 66 while
Queue Status
Queue Status will show the current status of the upload queue.
Disk Status History
If the number of bad sectors or available spare of the disk changes, the number will be recorded in Disk Status History. You can use this option to monitor the health of the disk.
IP Addresses
This will list network interfaces, including statistics. valid_lft=forever means the IP address is assigned statically. Otherwise, it indicates the remaining time of a DHCP Lease.
Network Configuration
Network Configuration shows either the response from the DHCP (Dynamic Host Configuration Protocol) server or static network configuration. A valid IP address is often the first thing to check for any network-related issues.
Below is an example of host configuration by Dynamic Host Configuration Protocol (DHCP).
TIME shows the time of configuration is done.
LIFETIME shows the lifetime of the configuration. Before LIFETIME expires, it will automatically obtain a new configuration (or confirm the current configuration is valid).
Dynamic Host Configuration Protocol allows setting DNS, NTP (not in the below example), and also static ROUTES table.
The ROUTES option (option 121) is shown with each record separated by space. Each route record is a target network (in CIDR) followed by a comma and gateway.
Timesync (NTP Time Synchronization) Status
This time synchronization command checks the current NTP server and its accuracy. Cloud security relies on an accurate clock on the CacheDrive. When startup, CacheDrive syncs the time with the Morro server (not relying on NTP), so time sync will always work initially. However, if your system or network configuration has issues, it will stop working after the clock skew gets large. CacheDrive uses the standard NTP protocol to sync, which can be configured through the standard Dynamic Host Configuration Protocol, or fall back to one of the public servers. If time sync is not successful, an alert email will be sent.
It is recommended to configure the DHCP server (on the AD Server) to enable the NTP option and use the AD Server as an NTP server. But you can use an external NTP server, which is the default behavior when the DHCP service does not send the NTP option. The Timesync service will use the NTP Servers provided by the DHCP server first. If multiple NTP servers are provided by the DHCP server, then the CacheDrive will contact each in turn until it is successful. If none of them connects, it will use the default fallback NTP servers. Fall back NTP servers are from the NTP pool, so the actual IP address may be different from time to time.
NTP Fallback (Used only when DHCP server does not send NTP): time.google.com 0.debian.pool.ntp.org 1.debian.pool.ntp.org 0.pool.ntp.org 1.pool.ntp.org 0.arch.pool.ntp.org 0.jp.pool.ntp.org
If using the default fallback NTP or the DHCP-specified external NTP service, port 123 (UDP) must be open. If DHCP specifies internal NTP service, it is your responsibility to make it accurate. The CacheDrive operation relies on an accurate NTP service.
Below is an example of when the device is using the Stratum 2 server. The polling interval ranges between 32 secs and 2048 seconds, depending on drive/jitter values.
Network Ping
Network Ping can check routability to the destination using the ICMP protocol. (Please note that Windows uses UDP to ping as default, and does not respond to ICMP Ping.) You can specify either an IPv4 address or FQDN.
Network Traceroute
Traceroute is similar to Ping, and it uses ICMP protocol with TTL. We search up to 30 hops. If you want to check the logical distance to an S3 bucket, you may enter for example, s3.us-west-1.amazonaws.com, and see how many hops there are. If more than 25 hops, you may want to consider using a bucket closer to your geographical location if possible.
Network Routing Table
The routing table shows how to route the network traffic. Using static network configuration, you can set only the default route. If you use Dynamic Host Configuration Protocol, you can set a complex route table to your needs. Below is an example, which set a complex routing table by the DHCP service.
Network Renew DHCP Lease
If you use DHCP and change some options on the DHCP server, you can force renew without rebooting or unplugging/plugging the cable. This is equivalent to Windows' ipconfig /renew.
Network Flush DNS
DNS or Domain Name System is a hierarchical and decentralized name resolution system. CacheDrive may cache the results based on DNS settings. However, sometimes you want to Flush DNS to quickly reflect new settings in the DNS service on your premises. This is equivalent to Windows' ipconfig /flushdns.
Network DNS Cache Statistics
You can check the statistics of the DNS cache.
Network Resolver Status
Show DNS name resolver status, including server address. If you change the server address in the DHCP server, you can renew the DHCP lease, and check the DNS name resolver status again.
In the below example output, the DNS server currently used is also the AD server. In case that server cannot be used, one of the Fallback DNS Servers is used. Please pay special care if you use a local DNS server. These must be able to resolve Morro's cloud address and other third-party services, such includes AWS, Wasabi, etc. If your AD domain is local (either using .local or not), external DNS cannot solve your Realm. So please pay special attention.
Network Resolver Query
You can send a query to the resolver. CacheDrive uses a built-in stub resolver as cache. If the answer is obtained from the cache, it is indicated as Data from: cache. You can specify FQDN or hostname (if the search domain is set properly by DHCP). Also if you specify an IPv4 address, then it will perform a reverse lookup.
Please note that international domains (IDNs) should be converted using Punycode to xn--{something}.
Network iPerf3 for one client
If you want to diagnose or benchmark your LAN performance between your client host and CacheDrive, you can run the iPerf3 server on CacheDrive. The server will be terminated automatically after one client is served.
Please note that iperf3 is not backward compatible with iperf2. So you need to install iperf3 on your client PC. Iperf3 does not officially support Windows, however, we have tested that it works with CacheDrive.
Queued Events by user
This diagnostic command examines the upload queue and counts the queue events per user. It shows the Number of Events and the Oldest Event by each User. The oldest column shows the timestamp (in the Device's local time) of the event.
Queued Events by share
This diagnostic examines the upload queue and count per share. It shows the Number of Events and the Oldest Event by each Share. Each share may be prioritized depending on whether RTC is enabled or not.
Heads of Queued events
This diagnostic show the first 10 events for file/folder operations in the Sync share, not including locks.
Heads of Queued events (with Priority)
This diagnostic show the first 10 events for file/folder operations in the Sync share, ordered by priority.
Largest Queued File
This diagnostic shows the top (largest) events in the queue for file upload (create or update). Some tools, such as ZIP, when saving files directly on a network share may add files one by one, thus unnecessarily generating multiple file uploads. For such applications, it is better to first create the ZIP file on a local PC/Mac, and copy to the CacheDrive.
Queued Files for Prefetch by Share
This diagnostic show the top (largest) 10 shares which in prefetch queue.
Heads of Prefetch Queued
This diagnostic show the first 10 files in prefetch queue.
Heads of Prefetch Queued (with Priority)
This diagnostic show the first 10 files in prefetch queue, ordered by priority.
ADS Machine Account Status
Joining to Active Directory will create Computer Account (a.k.a. Machine Account) in Active Directory. Using that Computer Account to authenticate to communicate AD server (or Kerberos Server). If there is the same Machine Account exists already, it may fail to join. After successful join, if Machine Account is deleted from the AD Server, the system may not work properly. To check the status of the Machine Account, you can use this command. Below is a sample output.
Send Wake-On-LAN packet
You can send a standard Wake-on-LAN (WOL) packet by specifying the target Ethernet Hardware address (MAC Address).
Diagnosis Helps
Why the CacheDrive cannot join the AD Domain
If you have experienced problems joining On-Premises Active Directory Domain (AD Domain), please check the following.
- Use "Resolver Query" diagnostics to see if your AD Realm can be resolved correctly. Please pay special attention if your realm is the same as the DNS domain. For example, your realm is "
mycompany.com", and then you need to have a local DNS server (typically the same as the AD Server), which should resolve the Realm to the local AD Server. But maintain public DNS entry to resolvemycompany.comto your website.- If you deploy in the cloud, you need to set up DNS entry in VPC (AWS) or vNet (Azure).
- Please check "Network Configuration" diagnostics, and see DNS entries. If there are multiple responses, all of them resolve your AD Domain properly. For example, if it returns 10.0.0.1 (your local AD) and 8.8.4.4 (Google-hosted public DNS), the resolver may use 8.8.4.4 to resolve names. Multiple DNS Server is used for load balancing as well as redundancy. You should not assume the second name server is used only while the first name server is down or not reachable.
- Use "Ping/Traceroute" diagnostics to see if the server is reachable. (You may enable AD Server to enable ICMP ping in the firewall.)
- If "Ping/Traceroute" does not work as expected, examine "Routing Table" diagnostics, in case your AD server is not in the same LAN segment as CacheDrive.
- In rare situations, you may need to check the service endpoint in DNS. You can use "Resolver Query". Important ones are
_ldap._tcp.yourDomainand_kerberos._tcp.yourDomain. Please consult with Microsoft documents for further information. - Check time is synchronized between the AD server and CacheDrive. You can use "Timesync Status" diagnostics to check. It is recommended to configure the DHCP server (on the AD Server) to enable the NTP option and use the AD Server as an NTP server. But you can use an external NTP server, which is the default behavior when the DHCP service does not send the NTP option.
- If it suddenly not working, please check "AD Machine Account Status". I the machine account (Computer Account) is deleted on the AD server, it will not work and need to rejoin.
- When joining a domain, and if the machine account with the same hostname already exists, the CacheDrive may not be able to join. In such a case, delete the machine account (Computer Account) on AD and try to join again.