Forum Replies Created
-
AuthorPosts
-
codingadvocateParticipant
Have you watched the screen captured videos on the website? The last video listed on the discovery page (https://www.opencontentplatform.org/discovery) provides a detailed walk through with how discovery works. Maximize the window and set the playback to high… about 20 minutes and you will have a good start.
I’m not aware of docs to describe how to create and deploy new jobs, but I can attest that the code is there.
If you want help with new job development/training/etc, talk to CMS Construct or others for paid services. The OSS side is for platform support.
codingadvocateParticipantHi Shwetanjali,
Clients:
OCP clients are instances of the OCP client software. If you walk through the REST API, you will find that a GET on ocp/config describes a GET on ocp/config/client as the following: “Report all server endpoints running one or more OCP service clients.”As such, the ocp/config/client/MyDesktop works if MyDesktop is running one of the OCP clients (e.g. contentGathering, universalJob, resultsProcessing). This is showing you runtime information tracked by instances of the client, for the purpose of system health, logging, self-restarts, etc. That shows OCP clients, not the discovered endpoints.
If you want to see data on all discovered objects, then use a different REST endpoint. For just Node context, you can do a GET on ocp/data/Node… or ocp/data/Windows or ocp/data/Linux, etc. Yes, a custom query in /query/this will do it, so you get related objects to the Nodes. But to see what the Find jobs gather, there is a view already packaged… called “Find endpoints” (GET on “ocp/query/simple/Find endpoints”). Although that looks more suited for a graphical view via admin console, since it’s not pulling all data on the Node objects.
Sticking with this idea: a GET on “ocp/query/config/simple/Find endpoints” shows you the query definition, where specific attributes are listed. You can save that as a new version, or send that definition through /query/this for an add-hoc query. I’d suggest removing the named attributes from those sections (leaving the list empty) so it pulls all attributes, and I think that will get you closer to your goal.
Discovery:
The packaged Find jobs pull high level OS/HW data like platform, version, hardware. Those connected objects are shown by the Find endpoints query just mentioned. If you want to get CPU/Memory/Disk info from all discovered endpoints, I don’t remember seeing anything specific to CPU/Memory type data. So you may need to create a job to go after your specific data points. Obviously that’s something the platform is enables.There are a variety of pre-packaged jobs to exemplify the types of directions you can go with the product. This is especially true with the dynamic discovery jobs. I haven’t seen another vendor product (paid or free) come close in comparison to the dynamic software discovery, which finds software mapped across servers – without any agents or software signatures. That one really helps with discovery efforts that feed Software Asset Management (SAM) projects.
codingadvocateParticipantHi Shwetanjali,
You can walk through the REST API by starting with the top and continuing down.
If you are looking for additional help with creating queries/reports/integrations for your use cases, I’d direct you towards CMS Construct (professional service provider for OCP). They provide a starter guide for the REST API under the downloads section of their ITDM page: https://cmsconstruct.com/itdm
codingadvocateParticipantTo summarize:
1) the OCP platform is running
2) the OCP API responds as expected
3) you’re seeing errors on a contentGathering job that attempts to connect to endpoints via PowerShell
4) this requires system administration troubleshooting with how PowerShell is setup/enabledIf you just enabled firewalls/PS access to enable IPs, then reset “connectViaFQDN” to false and use IPs (the default).
Also, don’t try remotely connecting to the same system. Windows security policies (from about 2 years ago now) mostly prevents that type of activity – on purpose. So if you’re going to test externally, open a PS session on the 192.168.0.85 machine (or wherever you’re running the contentGathering client)… and issue the New-PSSession to a different target endpoint (different IP).
This is where my help stops on this topic. The logs will tell you what is happening, so leverage them and relay that to a sys admin. Feel free to open new posts when you find and fix the PowerShell setup.
codingadvocateParticipantFrom what I can tell, you’re using name in the manual connection but using IP in the OCP connection. If you test with “-ComputerName 192.168.0.85” then you should see the message OCP is showing you.
To use IPs, make sure you have setup TrustedHosts to enable IPs. To use names, go to Platform->Configuration->OS Parameters in the admin console. Select Windows, and change the “connectViaFQDN” to true. OCP uses IPs by default since DNS isn’t always reliable at companies, but it can attempt name resolution if you tell it to.
Corporate IT usually has Group Policies or other automated means to globally set the PowerShell settings for PowerShell usage. But if you want to test it (again this is sys admin troubleshooting and not OCP troubleshooting – so here’s a freebie for you)… to setup IP-based connections:
====================================================
On Windows 10 Pro I just walked through this by disabling the Windows Firewall for both “Domain Profile” and “Private Profile”.Then using Norton Security as my firewall, I added a port traffic rule to allow local port connections in on 5985 (HTTP) and 5986 (HTTPS) for PowerShell default ports.
To ensure the user’s group has rights to use powershell:
Set-PSSessionConfiguration -Name Microsoft.PowerShell -ShowSecurityDescriptorUI -ForceRight click on PowerShell and ‘Open as Administrator”:
> WinRM quickconfig -force
> Enable-PSRemoting
> Set-Item WSMAN:\Localhost\Client\TrustedHosts -Value “192.168.0.*” -ForceThen to view the Trusted hosts:
> Get-Item WSMan:\Localhost\Client\TrustedHosts
=======================================And then to manually test:
=======================================
> $c = Get-Credentialcmdlet Get-Credential at command pipeline position 1
Supply values for the following parameters:
Credential> $s = New-PSSession -ComputerName “192.168.0.85” -Credential $c
> Invoke-Command -Session $s -ScriptBlock {hostname}
<remote_server_name>
> Remove-PSSession $s
=======================================codingadvocateParticipantTake a look at the input parameters on the job. When the portTestBeforeConnectionAttempt is set to true, if the socket connection fails to verify the ports (listed in portsToTest) are open, then it doesn’t attempt a remote powershell session.
The current error says, “Endpoint not listening on ports [5985, 5986]; skipping PowerShell attempt.” And that’s what it did.
Your previous error message was powershell specific: “WinRM client cannot process the request. Default authentication may be used with an IP address under the following conditions…”. You will see the same thing if you test outside of OCP.
As I suggested before, this is a system administration task. Looks like you need to enable PowerShell (or PS Remoting) such that you can remotely connect from the contentGathering client machine – to those target Windows servers.
codingadvocateParticipantGood to hear you have the platform up and running, and now basic jobs running.
The first error you showed is with the remote PowerShell connection to an endpoint (192.168.0.38). You can troubleshoot outside of OCP by opening PowerShell on the machine where you ran the contentGathering client, and issue a remote PS session to the target endpoint. You should see the same error as you showed above. This becomes a system administration type troubleshooting session, on whether the OS is enabled for PS remoting, whether to enable trusted IPs or DNS, etc.
You mentioned you had 63 other endpoints connected. I’d suggest using one of those IP’s for the config group job. You won’t be able to run that job on an endpoint that does not yet have a connected shell (like on 192.168.0.38), which is what that second error message says above (Could not find endpoint 192.168.0.38 with a corresponding shell. Please make sure to run the corresponding Find job first).
codingadvocateParticipantYou have the platform running, so that’s where my help stops. I’ll leave it to you to get a feel for how the system is working, and where you need to look for what.
But here’s a start:
===================
The server side logs (./runtime/log/service/). For example with the contentGatheringService.log, you will see communication with kicking off jobs and what clients are connected in, etc.Depending on the version of OCP, if you set ‘createExecutionLog’ to true on the job, you can have a job endpoint-specific log sent back to the server, under the ./runtime/log/job/<jobName> directory.
The client side:
The contentGathering client logs (./runtime/log/client/) [and similarly the universalJob type logs]:
contentGatheringClient.log (the top level log, where you will find server-to-client communication)
contentGatheringJobDetail.log (detailed info of job runtime; you can increase this per-job with the ‘printDebug’ parameter)
contentGatheringJobStatus.log (high level stats on job completion)The resultProcessing client logs (./runtime/log/client/):
resultsProcessingClient.log (info from data parsed off kafka flows from the clients)codingadvocateParticipantRegarding: “and then cmd.exe just exits”
I first read that, thinking you meant it was throwing an exception and the window was closing. But a second read of that can also mean the command exists after running platformConfig, and you are NOT getting an error.
If that’s the case, then you’ve already (successfully) run platformConfig, and you’re trying to just start OCP.
Regarding: “So when ever I try to start OCP by giving …framework\lib\platformConfig.py”
If you’re trying to *start* OCP, which seems like you might be saying above – that’s a different command. You want to run openContentPlatform (framework\openContentPlatform.py instead of framework\lib\platformConfig.py). And once OCP starts, you will be able to talk to the API (like you mentioned with Postman).
Hope that helps.
codingadvocateParticipantHello.
I’ll need more details before I can suggest more. To find those, take a look at your logs (runtime/log directory).
FYI, at that point in the platform configuration, packages are trying to be loaded into your database. So first I would verify that Kafka and the database both running. And that you’re able to talk to both kafka and the database (firewall rules/ACLs/proper certs if you’re using them/etc), that the right information was provided when configuring the database connection. Ensuring there were no mistypes when you gave IPs or server names, credentials, etc. You can review the information previously provided by looking in the config files (globalSettings and databaseSettings, in this case).
For what it’s worth, when installing/setting up on Windows, I prefer a PowerShell window instead of a cmd.exe shell. I never liked the whole disappearing shell feature, when the window closes after a script or app throws certain types of exceptions. I never found that very helpful. 🙂
codingadvocateParticipantAwesome. Good to hear you got it working on your laptop.
And thanks for the syntax note on initializeDatabase.py. It must have been fixed over the last few months, since I just checked and it’s not an issue in the current mainline (master branch). But we are due to roll another source drop through a testing and release cycle soon, for an updated “release” package.
Regarding the new question on the Postgres connection, please start a new discussion thread on that since this one has a couple topics covered already.
And, I’m assuming you used an actual IP Address for the database server. Meaning, if you look at your ./conf/databaseSettings.json file, you would see a line like this:
“databaseServer”:”10.11.12.13″
and not:
“databaseServer”:”IP”In the mean time, I can look to see if we’ve hit that before.
codingadvocateParticipantThe error you showed before suggested either OCP was not running, or OCP was configured with a different set of settings. Specifically, the Python requests library couldn’t connect to localhost:52707.
Is OCP running, when you attempt the admin client connection? If so, did you try to use a 3rd party REST API browser (Insomnia, Postman, etc) instead of the admin console, to verify the connection to the OCP API?
The previous error showed the connection was being made to localhost. If that’s the case, make sure you run the admin console on the same server as OCP (for the loopback address to work). If you have it running on another machine, then make sure the REST endpoint <server_name_or_ip_address:52707> is reachable… ensure no network or software firewall rules or ACLs are blocking it.
codingadvocateParticipantHello again.
In short, the admin console isn’t able to talk to the OCP API. This all depends on how you setup OCP. The error is saying it can’t connect to ‘localhost’. Troubleshooting suggestions follow.
First, make sure OCP is running (python openContentPlatform.py) either on the command line or as a service/daemon – before trying to use the admin console. From the error, it looks like you’re running on Windows. Here’s what it should look like when the connection works:
> python .\adminConsole.py
Checking browser emulation setting in registry
Key already found with value: (11001, 4). Leaving key unchanged.
<now a window opens with the admin console>You can use a 3rd party REST API browser (e.g. Insomnia, Postman), to verify the connection to the OCP API. The admin console does the same thing those browsers would to… communicating to the OCP REST API.
You mentioned updating the ocpSettings.json file with a valid apiUser and apiKey. Great. Now make sure the other ocpSettings match what you provided when you configured OCP. To do this, in the admin console directory, open ./conf/ocpSettings.json… and in the OCP directory, open ./conf/globalSettings.json. Go through the admin console settings and make sure they match what you configured for the OCP setup. The values for the following mapping should match:
[admin console setting] == [OCP globalSetting]
“restEndpoint” == “apiIpAddress”
“restProtocol” == “transportType”
“restPort” == “apiServicePort”
“restPath” == “apiContextRoot”If you setup SSL, you need a cert (ocpCertificateFile). Etc.
codingadvocateParticipantHappy to help. I’m assuming that is the path where you put it, right?
You mentioned you are trying to run that on cmd (assuming cmd.exe), and I believe the sample commands were shown for a PowerShell session. Either should work. But you may need to double quote the command arguments. Maybe something like:
python “D:\Software\openContentPlatform\framework\lib\platformConfig.py”
codingadvocateParticipantThe API user/key are generated when you install OCP.
For reference, about half way through the “Installing OCP” section on the install page (https://www.opencontentplatform.org/install), you run platformConfig.py. That’s where the base API user/key are created. You can certainly create more, to extend to different external API users, after installing OCP. This can be done either through the admin console (which you’re trying to setup), or through an API call directly.
-
AuthorPosts