vCommander® receives updates about VM resources continually from locally connected cloud accounts (managed systems), and on a defined schedule for public cloud accounts (managed systems) (changes originating in vCommander are updated immediately). This keeps the information available to vCommander users as accurate as possible at all time.


Problem

However, in larger, more active environments, it’s possible that a very high number of updates coming from an on-premise cloud account (managed system) can have a detrimental effect. For example, if thousands of VMs are reporting insignificant disk space changes (lower than 1 MB), so much information is available that updates begin to be dropped to protect overall system performance.


Typical environments will never experience this issue.


Symptoms

Errors similar to the following will be logged in the <Install_Directory>\Embotics\vCommander\tomcat\log\vcommander.log file when updates are dropped:


2014-10-15 05:50:04,644 [VIMMonitor-your_vCenter.company.com] ERROR - Unable to add observation group to queue (size 10000) (thread state RUNNABLE) (handler progress LOCATE)

2014-10-15 05:50:04,644 [VIMMonitor-your_vCenter.company.com] ERROR - Dropped Observations due to full queue: ObservationPack type=OBSERVATION group=[ModificationObservationGroup: 1 observations from [MANAGEMENTSERVER 163840 your_vCenter.company.com] ]:


Additionally, users may notice a lack of synchronization between vCommander and the cloud account (managed system). Updates being dropped may be insignificant, like the minor disk space changes already mentioned, or may be very significant, like the fact that a VM deployment has completed.


Solution

vCommander includes a mechanism by which resource updates may be throttled to prevent this from happening. Throttling the observations has the same effect as extending the Update Frequency for public cloud accounts (managed systems); any changes originating in vCommander are updated immediately, but changes originating from the cloud account (managed system) are updated a little bit later.


Turn on this throttling by following the steps below:

  1. Open the file <Install_Dir>\Embotics\vCommander\tomcat\common\classes\vim-throttled-properties.conf using a text editor.

  2. Add one or more lines below to the file, depending on the resources frequently changing, then restart the vCommander Windows Service:

    COMPUTERESOURCE=totalProcessors
    COMPUTERESOURCE=totalMemory
    COMPUTERESOURCE=totalCpu
    COMPUTERESOURCE=cluster



    If you need assistance in determining which throttling to turn on, please contact Embotics® Technical Support.


You can view the number of observations in the queue by navigating to the link below and authenticating as a super user:


https://<vCommanderFQDN>/webservices/services/rest/v2/monitoring/observation/queue




See Also