1 / 22

Dynamic Scheduling Using Pools and Dynamic Pools in Dynamic Domains

TWS Education. Dynamic Scheduling Using Pools and Dynamic Pools in Dynamic Domains. Agenda. Overview of dynamic scheduling Dynamic Agents Pools Dynamic Pools Troubleshooting Migration Integration Troubleshooting Architecture Database changes. Overview.

kory
Download Presentation

Dynamic Scheduling Using Pools and Dynamic Pools in Dynamic Domains

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. TWS Education Dynamic Scheduling Using Pools and Dynamic Pools in Dynamic Domains

  2. Agenda • Overview of dynamic scheduling • Dynamic Agents • Pools • Dynamic Pools • Troubleshooting • Migration • Integration • Troubleshooting • Architecture • Database changes

  3. Overview • Distribute, balance and optimize workload to “best available” resource across dynamically shifting, cross-enterprise resource pool based on resource utilization • Route jobs to any available node that matches the resource requirement • Discover newly available servers as part of the pool and use them as possible job targets

  4. MDM Controller Agent Dynamic Agents TWSd and TWSz are sharing the same distributed agent. This agent can run “static” jobs (jobs defined on this particular agent or “dynamic” jobs. This agent is monitored and you can check the status. This agent can run job of several types.

  5. MDM Controller Pool or Dynamic pool Pools and Dynamic pools • Dynamic agents can be grouped in pools or dynamic pools • A pool is a set of agents identified by a list of workstation • A dynamic pool is a set of agents identified by a set of requirements • This agent can run job of several types. • Pools are monitored (they are considered as not running if all the agents belonging to the pool are not running). • The load is balanced among the pool members • The load is sent to the machines that are currently running • Different policies can be used while defining dynamic pools to drive the load to the appropriate machine

  6. Dynamic agents • Dynamic agents are TWS workstations • Automatically defined • Automatically updated (in case of address/port change) • Modify the agent configuration and the change is reflected in the workstation definition • This update is immediate, no need to recreate the plan • Added to the plan (can define job, job stream, limit, fence...) • Monitored (linked when the broker server is linked, running when the dynamic agent is running) • Can run all job of any type (including “docommand”) • Can use “Windows users”

  7. Components: Dynamic Agent definition • The dynamic agent is automatically defined CPUNAME MY_Agent DESCRIPTION "This workstation was automatically created" OS UNIX NODE ucasell2.romelab.it.ibm.com SECUREADDR 51114 TIMEZONE Europe/Rome FOR MAESTRO HOST UCASELL2_DWB TYPE AGENT PROTOCOL HTTPS END COMPOSER DEFINITION OF A WORKSTATION OF TYPE “Agent”

  8. Pools • Pools are TWS workstations • A pool is a set of TWS dynamic agent workstations (or remote engines) • The load is distributed among the workstations belonging to the pool • A change to the pool definition is immediate (no need to update the plan) • Added to the plan (can define job, job stream, limit, fence...) • Monitored (linked when the broker server is linked, running when at least one agent is running) • Can run all job of any type (including “docommand”) • Can use “Windows users”

  9. Components: Pool definition • The pool is defined by using composer or the TDWC CPUNAME MY_Pool DESCRIPTION "Sample pool" OS UNIX TIMEZONE Europe/Rome FOR MAESTRO HOST UCASELL2_DWB TYPE POOL MEMBERS UCASELL2_1 UCASELL3 END COMPOSER DEFINITION OF A WORKSTATION OF TYPE “pool”

  10. When to use pools • High availability scenarios • Load balancing scenarios • Clusters • Sysplex

  11. Dynamic Pools • Pools are TWS workstations • A pool is a set of TWS dynamic agent workstations defined with a set of requirements • “Physical” requirements (OS, architecture, CPU usage...) • “Logical” requirements (logical resources associated to the workstations) • “Global” requirements (logical resources available in the environment) • The customer can define the optimization policy base on • Number of jobs • CPU usage • Logical resources quantity • Logical resources utilization

  12. Dynamic Pools • It is possible to define an ordered list of workstations • A change to the pool definition is immediate (no need to update the plan) • Added to the plan (can define job, job stream, limit, fence...) • Monitored (linked when the broker server is linked, always running) • Can run all job of any type (including “docommand”) • Can use “Windows users”

  13. Components: Dynamic Pool definition CPUNAME MY_DPool DESCRIPTION "Sample dynamic pool" OS UNIX TIMEZONE Europe/Rome FOR MAESTRO HOST UCASELL2_DWB TYPE D-POOL REQUIREMENTS <?xml version="1.0" encoding="UTF-8"?> <jsdl:resourceRequirements xmlns:jsdl="http://www.ibm.com/xmlns/prod/scheduling/1.0/jsdl"> <jsdl:resources> <jsdl:logicalResource quantity="40" subType="type01"/> </jsdl:resources> </jsdl:resourceRequirements> END • The dynamic pool is defined by using composer or the TDWC COMPOSER DEFINITION OF A WORKSTATION OF TYPE “dynamic pool”

  14. When to use dynamic pools • High availability scenarios • Complex load balancing scenarios • Groups defined by resources • Complex limits on jobs (for example licenses constrains) • Very dynamic environments (machines provisioned/de-provisioned often)

  15. Dynamic scheduling in 8.5.1 vs 8.6

  16. Troubleshooting workstations (1/2) • Unlinked agent, pool or dynamic pool • All the workstations hosted by the broker server are unlinked if the broker server is unlinked • Dynamic agent not running (no “J” on conman) • There is no persistent connection to dynamic agents, there is a heart-beating mechanism. The configuration is in the ResourceAdvisorConfig.properties file #Specifies the time interval in seconds within which the Resource Advisor expects a heartbeat signal from the Workload Agent. The value defined in this parameter must be consistent with the NotifyToResourceAdvisorIntervalSecs parameter defined in the ResourceAdvisorAgentConfig.properties #file, which defines the time interval for the Workload Agent to send the heartbeat signal to the Resource Advisor. RaaHeartBeatInterval=367 #Specifies the number of missed heartbeat signals after which the computer is listed as not available. MissedHeartBeatCount=2 • Pool not running (no “J” on conman) • Check the agents belonging to that pool

  17. Troubleshooting workstations (2/2) • The log file for error message in the heartbeating calls on the agent is “JobManager_message.log”. When working fine the following log entry is present:AWSITA083I Resource information was sent to "https://ucasell2.romelab.it.ibm.com:31116/JobManagerRESTWeb/JobScheduler/resource" • The URIs of the broker server (and its backups)are sent by the broker server itself and stored in the JobManager.ini file:BackupResourceAdvisorUrls = "https://localhost.localdomain:31116/JobManagerRESTWeb/JobScheduler/resource","https://ucasell2.romelab.it.ibm.com:31116/JobManagerRESTWeb/JobScheduler/resource" • The customer can customize these URIs by using the importServerData and exportServerData commands

  18. Troubleshooting Jobs • In case of jobs in the “fail” state the “stdlist” will show the appropriate error message. • In case of abend or any unexpected job result, additional information can be found in a directory like the following: “/opt/IBM/TWA86/TWS/stdlist/JM/2011.05.16/archive” where a zip file contains information about the job. The information includes • The job definition • The job output • The parameters passed to the taskLauncher • The trace file of the taskLauncher for that job • The name of the workstation that actually ran the job is in the header of the stdlist • The job definition on the broker is a JSDL. This JSDL is created behind the scenes by the broker, using the TWS job definition and the TWS workstation definition. This JSDL is reported in the header of the stdlist

  19. Starting and Stopping the Workload Broker • Starting dynamic workload broker • Use startBrokerApplication.sh on UNIX and Linux or startBrokerApplication.bat on Windows as follows: startBrokerApplication -user username -password password [-port portnumber] • where username and password are the credentials used during the installation. • The parameter portnumber is optional. If it is not defined, thedefault is used. • Stopping dynamic workload broker • Use stopBrokerApplication.sh on UNIX and Linux or stopBrokerApplication.bat on Windows as follows: stopBrokerApplication -user username -password password [-port portnumber] • where username and password are the credentials used during the • installation. • The parameter portnumber is optional. If it is not defined, the default is used. • Run the link command. • If you do not run this command, the dynamicworkload broker server is automatically linked ten minutes after the stop operation. • Displaying dynamic workload broker status • Use brokerApplicationStatus.sh on UNIX and Linux or brokerApplicationStatus.bat on Windows as follows: brokerApplicationStatus -user username -password password [-port portnumber] • where username and password are the credentials used during the installation. • The parameter portnumber is optional. If it is not defined, the default is used.

  20. Migration • The TDWB jobs can still be executed and so a migration effort is not required • The “JSDL template” workstations defined as XA on the TDWB server are still supported, even if customers are encouraged to use pools or dynamic pools. • Most of the jobs running on FTAs or Standard agents can run on a dynamic agent (or pool or dynamic pool).

  21. Dynamic Scheduling architecture Symphony MDM BMDM Symphony Broker Symphony Symphony DM Backup Broker BDM There is one broker database in each domain The green arrows are for Restful web services Symphony FTA Agent Engine

  22. Dynamic Domain Managers • DDM is a new installation bundle • DDM is common between D and Z • A DDM can be shared by TWSd and TWSz • On D: • the Domain, the domain manager and the broker workstation are automatically created an kept up to date • the Backup DDM is automatically configured (by accessing the same database of the DDM. • On Z: • with DDM you can easily use dynamic scheduling • The user scenarios to define pools, dynamic pools and jobs are the same on TWSd and TWSz (same panels on the TDWC) • The failover of the agent from the DDM to the BDDM is automatic • Each DDM can serve a very high number of agents (more than a DM).

More Related