CitectSCADA Virtualized Environment

I would like to think that some people have implemented Citect Servers in a Virtualized environment  HyperV  , VMWare ESX   etc and I would like to get some feedback on how it all went and if you have any nuggets of wisdom to offer.

  • Hi Richard, I believe it's very common. We installed on VMWare ESXi for another customer with multiple sites. The only gotchas are in relation to poor virtual infrastructure management rather than the product itself. IT management of the server gives neither you nor the operational customer adequate visibility of the host's allocation and performance. IT people don't understand the criticality of OT systems, and think nothing of underspecifying a host machine and running too many VMs on a host with a large performance impact. Essentially the VM should act just like you've installed it on bare metal. If it just seems unresponsive and sluggish, then the host is probably overloaded.
    Utilities like ESXTop can help to identify the exact bottlenecks if need be.
  • This is very true. Also, some IT staff don't understand the inner workings of CitectSCADA. Unfortunately it is essentially still a single thread application, so it benefits more from having high clock speeds rather than the number of CPUs. IT usually think they can fix application performance by throwing more vCores or RAM at it, but this will not work with Citect.

    I find that Citect runs great on small-office servers that have only a few CPU cores and high clock speeds, and performs a lot less on the standard infrastructure that IT uses for virtualization of office applications. For the most demanding Citect projects we use dedicated hardware with just one or two VMs running on it. Virtualization is then only used for the ease of maintenance (hardware upgrades).

    If you want to run your whole OT environment on a single (redundant) server with many CPU cores, you may end up buying very expensive CPUs.

  • Hi Richard,

    Implementing the licenses is a thing you should plan well. You can use hardware dongles or softkeys, but both solutions have some caveats in a virtual environment.

    It is not advised to plug a hardware USB dongle directly into a VM server, because it will not be available for a VM anymore after a hardware redundancy failover or planned shutdown of the primary hardware (the VM then continues to run on a different physical machine). Most people mitigate this issue by using network USB hubs, like USBAnywhere.

    Softkeys can be installed in virtual machines, but you have to be careful not to corrupt them. Some IT scenarios can break the license, like restoring a VM from a back-up or snapshot, copying or moving the VM, changing network configuration (MAC address), etc.

    Best regards,
    Patrick
  • hi Richard,
    We have a big number of sites with virtualized environments (in our case HyperV) and be careful with the VM, if you decide to go with soft licenses:
    - As Patrick says there are multiple scenarios where the license can be broken (we have suffered it)
    - Also we are suffering other problem with soft licenses in Citect 2018 (still under investigation with Aveva) that it is lost randomly (several times per day) and Citect gets restarted

    BR

    ANA

  • Richard,

    I have a client running Citect2018R2 on HyperV, we used hardware dongles and software called Kernelpro USB over ethernet (https://www.kernelpro.com/) to "map" the USB dongle from the host (WinServer 2019) to the Citect HyperV VM. It works very reliably and is not expensive with a one time licence fee.

    Tony
  • As others have stated, USBAnywhere seems to work well if you've got hardware dongles.

    WRT soft licensing issues, licenses tend to be based on a combination of the hard drive device ID, network cards and their MAC addresses, computer ID etc. In a hardware setting this stuff doesn't change. Most likely of these to change in a virtual setting is the MAC address if it hasn't been set to static. This is a great proactive step you can take in specification stage: STATIC MAC ADDRESSES.

    Hyper-V in particular does MAC addressing poorly (IMO). All servers start with the same default virtual dynamic MAC range, and you need to go and change them so they don't clash. Out of ignorance I hadn't done this and the result was something called "MAC Flap" on the switch, where the MAC will be registered to one physical port, then switch to another one and back to the first in perpetuity.
  • Nice to share.
    Note: This is not supported by Citect so if you have any issue you are on your own!
  • I tend to agree with Patrik feedback but there a lot of sites that are running license in a virtual environment using Software dongles with no big issues and having the benefit of all the software functionality.
    Think about this: If I am in a datacenter, where should I plug my USB or AnywhereUSB device?
  • Hi Antonio,
    I don't have many issues with softkeys in VMs as well, as long as I follow the rules regarding back-ups, etc.
    Most of our customers use softkeys in VMs in their production environment, but use USB licenses in the test and development machines because these machines are more subject to changes, reboots and restores.

    About datacenters: I would not ever put SCADA servers in a datacenter, especially not Citect. It is called PLANT SCADA for a reason.
    I might consider it for SCADA clients or MES applications though.