Loading in 2 Seconds...
Loading in 2 Seconds...
Workflow ETL workflow Complex event processing(CEP ) Mona Alnahari. Workflow is the definition, execution and automation of business processes where tasks, information or documents are passed from one participant to another for action, according to a set of procedural rules.
Complex event processing(CEP)
Workflow is the definition, execution and automation of business processes where tasks, information or documents are passed from one participant to another for action, according to a set of procedural rules.What is a workflow
During the years, workflow technology has evolved from just workflow automation, which has been focused on automating human centric processes, through Enterprise Application Integration, which is focused on the exchange of information between heterogeneous systems, through Business Process Modeling and Analysis,
which is focused on gaining a detailed understanding of business processes and the potential impact of changes to those processes, through Business Activity Monitoring, which is focused on analyzing the efficiency and effectiveness of business processes and activities, to BPM that supports the complete process improvement cycle, from definition to implementation, monitoring and analysis, and through ongoing optimization.
The workflow software solutions introduced in the 90’s were based on workflow engines and business rules. Their main objective was to replace the paper-based task-routing activities with automated electronic form processes. By the late 1990s, the workflow software market integrated advanced functionality, including business rules engines, policy management, modeling tools, process monitoring, and optimization.Workflow Software History
in 1993, the Workflow Management Coalition (WfMC) is a global organization of adopters, developers, consultants, analysts, as well as university and research groups engaged in workflow and BPM. The WfMC creates and contributes to process related standards, educates the market on related issues, and is the only standards organization that concentrates purely on process.Workflow Software History
The WfMC created Wf-XML and XPDL, the leading process definition language used today in over 80 known solutions to store and exchange process models. XPDL is a process design format for storing the visual diagram and all design time attributes.Workflow Software History
In 2005, Microsoft deployed the Windows Workflow Foundation which replaced the basic workflow engines with sophisticated workflow functionality.
Today workflow software needs to comply with the full BPM discipline of modeling, execution, simulation, optimization and monitoring functionality.Workflow Software History
BPM is an integrated collection of critical software technologies that enables the control and management of business processes
BPM software is a model-driven workflow environment that makes the model executable, while keeping the model as the central focus for future process changes.BPM – Business Process Management
A workflow application is a software application which automates, at least to some degree, a process or processes. The processes are usually business-related, but it may be any process that requires a series of steps that can be automated via software.
In machine shops, particularly job shops and flow shops, the flow of a part through the various processing stations is a work flow.
Insurance claims processing is an example of an information-intensive, document-driven workflow.
Wikipedia editing is an example of a stochastic workflow.Workflow Examples
Some steps of the process may require human intervention, such as an approval or the development of custom text, but functions that can be automated should be handled by the application. Advanced applications allow users to introduce new components into the operation.
The Getting Things Done system is a model of personal workflow management for information workers.
In global software development, the concept of follow- the – sun describes a process of passing unfinished work across time zones.Workflow examples
In Service- oriented architectures an application can be represented through an executable workflow, where different, possibly geographically distributed, service components interact to provide the corresponding functionality, under the control of a Workflow Management System.Workflow examples
Workflow software provides organizations with the technology functionality, enabling them to innovate and create new working environments that not only define the business process and tasks, but execute and run them through common day-today applications like Outlook and Word.The benefit of workflow software
Improves productivity – Automated workflows reduce the time spent on manual tasks.
Visibility – Workflows enable managers to see what is happening with business critical processes at every point along the process, and at any moment in time.
Faster business reaction to change – Workflow software enables organizations to react quickly and smoothly to market changes through process modifications.The benefit of workflow software
Improves Accountability – Monitoring time to execute business functions creates productivity measurement and continual process improvementThe benefit of workflow software
Sequential Workflow (Typically Flow Chart based, progresses from one stage to next and does not step back)
State Machine Workflow (Progress from ‘State’ to ‘State’, these workflows are more complex and return to a previous point if required)
Rules-driven Workflow (Implemented based on a sequential workflow. The rules dictate the progress of the workflow)Three types of workflow
An ETL workflow is a design blueprint for the ETL process. The designer constructs a workflow of activities (or operations), usually in the form of a graph, to specify the
order of cleansing and transformation operations that should be applied to the source data, before being loaded to the data warehouse.ETL Workflows
Data warehousing is a technology that enables decision-making and data analysis in
To populate a data warehouse with up-to-date records extracted from operational sources, special tools are employed, called Extraction – Transform – Load (ETL) tools, which organize the steps of the whole process as a workflow
1. The identification of relevant information at the source side.
2. The extraction of this information.
3. The transportation of this information to the Data Staging Area (DSA), where most of the transformation usually take place.ETL workflow tasks
4. The transformation of the information coming from multiple sources into a common format.
5. The cleansing of the resulting data set, on
the basis of database and business rules.
6. the propagation and loading of the
data to the data warehouse and the refreshment of data marts.
ETL tools constitute a multi-million dollar market Due to their importance and complexity.
The traditional database vendors provide ETL
solutions along with the DBMS’s: IBM with InfoSphere Information Server , Microsoft with SQL Server Integration Services (SSIS) , and Oracle with Oracle Warehouse Builder .
There also exist independent vendors that cover a large part of the market.
Nevertheless, an in-house development of the ETL workflow is preferred in many data warehouse projects, due to the significant cost of purchasing and maintaining an ETL tool.
The spread of existing solutions comes with a major drawback. Each one of them follows a different design approach, offers a different set of transformations, and provides a different internal language to represent essentially similar functions.
Although Extract-Transform-Load (ETL) tools are available in the market for more than a decade, only in the last few years have researchers and practitioners started to
realize the importance that the integration process has in the success of a data warehouse project.
Event processing is a method of tracking and analyzing (processing) streams of data about things that happen (events),and deriving a conclusion from them.
Complex event processing, or CEP, is event processing that combines data from multiple sources to infer events or patterns that suggest more complicated circumstances. The goal of complex event processing is to identify meaningful events (such as opportunities or threats) and respond to them as quickly as possible.Complex event processing
A computation-oriented CEP solution is focused on executing on-line algorithms as a response to event data entering the system. A simple example is to continuously calculate an average based in data on the inbound events.
Detection-oriented CEP is focused on detecting combinations of events called events patterns or situations. A simple example of detecting a situation is to look for a specific sequence of events.CEP types
Workflows automate the flow of employee tasks and activities, reducing the time the process took to complete as well as potential errors caused by human interaction.
Workflows makes processes more efficient, compliant, agile, and visible by ensuring that every process step is explicitly defined, monitored over time, and optimized for maximum productivity. Given optimal, up-to-the-minute process data, managers and employees can take quicker action and make smarter decisions.
Workflows empower business users and IT to work together to rapidly modify systems and processes to reflect changes in the business.Summary