Plant SCADA 2023 OPC UA Server issues on domain connected machine - possibly group policy conflicts

Hi All. I am trying to utilise the OPC UA server in Plant SCADA 2023 on a machine that is connected to a domain. When I try and connect with an OPC UA client on the machine the connection times out.
Upon further investigation in the Windows security logs there are audit failures related to Senstive Priviliege Use via the NT Service\ConnectivityService. It appears that the Citect.Connectivity.Server.exe is trying to raise the SELoadDriverPrivilige and it cannot do so.
Has anyone seen similar behaviour? If I load Plant SCADA 2023 on a non domain connected machine I do not see this behaviour and I can connect to it via a client.

Parents
  • Hi  ,

    How did you connect the OPC UA server, using Anonymous or a domain account? Is the encryption enabled on the OPC UA server?

    If you use a domain account to connect to the server, the server will pass the credential to AIM (PCS Token Host/SMS) for authentication. AIM then sends a request to the domain controller for authentication and get a returned token. The token will be passed to SCADA IO servers for authentication and authorization (if writes are permitted). Take a look at Operational Control Logger (aaLogger) on the OPC UA server machine and see if any message is logged for the connection timeout. Also take a look at aaLogger on the System Management Server machine.

Reply
  • Hi  ,

    How did you connect the OPC UA server, using Anonymous or a domain account? Is the encryption enabled on the OPC UA server?

    If you use a domain account to connect to the server, the server will pass the credential to AIM (PCS Token Host/SMS) for authentication. AIM then sends a request to the domain controller for authentication and get a returned token. The token will be passed to SCADA IO servers for authentication and authorization (if writes are permitted). Take a look at Operational Control Logger (aaLogger) on the OPC UA server machine and see if any message is logged for the connection timeout. Also take a look at aaLogger on the System Management Server machine.

Children
  • Hi Jacky,

    To try and fault find further I ended up:

    1) Removing the machine from the domain so it is in a workgroup

    2) Uninstalled Plant SCADA and all components

    3) Reinstalled Plant SCADA and all components

    I still have the same issue. To answer your question I am trying to connect as Anonymous and have ticked the 'Allow anonymous access box' in the OPC UA server configuration. 'Enable encrypted communications' is not selected at the moment.

    With regards to looking at Operational Control Logger can you point me where to find that? I'm relatively new to Plant SCADA.

    Aside currently the Enpoint URL is opc.tcp://aurwrjescdfr01:48032/plantscada

    That is the port is set to 48032.

    One thing I have noticed is that if I go to the command prompt and type netstat -a 48032 is not listed as a listening port.

    When I do this on the separate laptop where I have installed Plant SCADA and an OPC client can connect I can see that port is listening when the same command is run. Something seems to be stopping it listening on the machine in question.

    Windows firewall is totally disabled.

    Unsure what else could stop the Aveva Plant SCADA Connectivity Service listening on that port?

    Finally it may or may not be related but if i run the Common Services Portal Troubleshooting SCAN everything comes up as okay except there is an issue coming up as follows:

    "An issue with Service endpoint discovery configuration detected. Software components may not be able to communicate with each other. For more information, see the “Repairing or Re-installing Platform Common Services” section of the AVEVATm Common Services Portal User Guide." 

    Note Plant SCADA has just been reinstalled so no idea how there could be an issue straight away?

    Any other assistance appreciated.

    Regards

  • Hi  ,

    • You can launch aaLogViewer.exe from C:\Program Files (x86)\Common Files\ArchestrA, the default location. Pin it to start for later convenience.
    • By default, Connectivity Service is configured to use port 48031. It is okay to change to use 48032.
    • The error you mentioned above indicates that PCS framework is in the faulty state. Technically, Connectivity Service is running on top of PCS framework. You need to fix this PCS issue.
      1. Try to restart "PCS Agent (Watch Dog) from Services.
      2. If the problem persists, launch Configurator.  Select System Management Server, then select the 3rd option (no SMS) and click Configure. Finally use option 2 to configure (SMS).
      3. The last option is to repair PCS framework from Control Panel. Rerun Common Services Portal Troubleshooting SCAN to ensure all components having the green tick. 

    Reinstalling Plant SCADA won't help because the issue is on the PCS side.

  • Hi Jacky,

    Thanks.

    I only change connectivity service to port 48032 to try a different to default, have changed back but no effect.

    I have been through steps 1,2 and 3 you have given however the issue is still present.

    I can now access the logger and export logs. Would sending a log possibly help diagnosis? Unsure I can do that through this forum?

    Aside unsure how important it is but some program revisions are:

    Aveva Plant Common Services 7.0 Version 7.0.22195.1

    Aveva Plant SCADA 2023 Version 8.40.525

    Windows 11 Pro Version 22H2 OS Build 22621.3155

    Regards

  • Hi  ,

    Connectivity Server is a standard Windows service running under a virtual service account. It should be in the run mode automatically once installed regardless a SCADA project is running or not. Have you checked Windows Event Viewer when Connectivity Service starts up? There could be an exception that prevents Connectivity Service from running properly. You could use the filter with .NET Runtime and Application Error in the event log (as shown below). Hopefully, you could find some useful information.


    I am not aware that Citect.Connectivity.Server.exe is trying to raise the SELoadDriverPrivilig. If the problem still persists, you should reach out to Technical Support for further troubleshooting.

  • Hi Jacky,

    Once the machine was moved off the domain there were no more errors in the windows event logs related to SELoadDriverPrivilig.

    I have started and stopped the Connectivity Service via Services and when I do so there are no new events being logged in the Windows Event log which makes me believe it is running correctly.

    I will reach out to Technical Support. Thanks for your assistance.

  • Hi Jacky,

    I think I have isolated the problem. When Plant SCADA was first installed the PC Machine name was FRWF-SCADA-PC-2. The machine name has since changed to AURWRJESCDFR01.

    Looking in the aaLogViewer there are the following errors:

    ERROR: Overriding ApplicationUri (urn:AURWRJESCDFR01) with URI from ApplicationCertificate (urn:FRWF-SCADA-PC-2).

    Domain (aurwrjescdfr01) used in URL does not match certificate ({LOCALHOST|FRWF-SCADA-PC-2}).

    It therefore appears that I need to change the Application Certificate to have the correct machine name (AURWRJESCDFR01)

    Do you know how I go about either changing the machine name in the certificate or re-issuing one with the correct machine name?

    Regards

    PS I have reached out to tech support I just have not heard back yet.

  • Certificates are managed by PCS. I don't think PCS likes it when the machine name is changed.

    I'd recommend uninstalling PCS (then check registry to make sure any reference to PCS and the old machine name is deleted), the install PCS again. This should clear up the issue you were facing.

  • Hi Olivier,

    I have uninstalled PCS however when I look in the certlm.msc there is still a certificate there as below. It

    It looks like that certificate has been there potentially since the packages were first installed last year by others. Uninstalling PCS has made no difference.

    Can I simply delete that certificate before I re-install PCS with the hope a new one will be created?

  • Hi  ,

    As Olivier pointed out, PCS framework currently doesn't support changing the machine name after installed. You can manually delete the old certificate. Reinstalling PCS should install a new certificate of OPC UA (you may need to restart the machine after reinstallation). If the certificate is still not accessible (the message should be reported in Logger Viewer), try the following steps.

    1. Launch Manage Computer Certificate and make a copy of the certificate thumbprint

    2. Launch a command line window as administrator

    3. Execute certutil -repairstore my [thumbprint]