Suggestions for improving Etlworks Integrator
45
votes
Packages and versioning
It would be nice to create a "package" of flows and connections and store it as a certain version. It should then be easy to switch between versions just by toggling ...
32
votes
2FA Login
Our security policy will require all tools that have access to company data, to have 2FA required to login.
Currently a user can login using only the username and ...
Done
12
votes
GUI Database Lookup
As a user, creating database lookups requires the user to know code in the field value function to perform database lookups. This is even more complicated when doing ...
12
votes
Drag and drop process builder for nested flows
Replace existing nested flow editor with drag and drop process builder:
- works similarly to the diagram editor
- ability to select existing flows from the side-bar ...
10
votes
Ability to update all or selected agents to the specific version
Currently each agents needs to be updated individually.
From our perspective, for bulk updating, I would say it would be managed via the Agents page itself.
- ...
10
votes
SSO Authentication
Etlworks handles and processes a lot of sensitive data. It is important that this is adequately shielded and becomes part of our company-wide security umbrella. The ...
10
votes
Retrieve Parent Context of a Flow on Error
We have a shared pipeline with a hierarchy of nested flows. When an error occurs in a deeply nested flow, our support team needs to determine its execution context ...
Under consideration
9
votes
Test Script button
Testing the „Execute JavaScript or Python code“ type flows is very troublesome, as you have to save (which closes the modal), press run, open the flow execution view, ...
9
votes
Drag&drop SQL query builder and Field Function Bilder
- SQL Builder (in Flow and Explorer)
- Field Function Builder (at least Lookup)
8
votes
Bulk edit configuration mapping
Instead of updating source/destination field names or data types individually, allow selection of desired fields for bulk modification(s).
Done
8
votes
Recycle Bin when deleting flows (or other assets)
It would be nice if there was a Recycle Bin for flows (or other assets), and it could have a time limit (like 30 days). This way, if one of our users accidentally ...
8
votes
In agent flow, I want an email to send when expected flow execution time field exceeded
In the agent flow, when going to the "Configure flow notifications", when I enter an amount of minutes for the "Expected flow execution time", it currently does not ...
Done
8
votes
(re-)Run from error screen
If investigate an issue within ETLworks. You go to the error (via status) to the statistics section. When solving the issue, you have to search for the flow in order ...
8
votes
Usability enhancement for Flows Builder
1. Ability to edit inner flow from the nested flow
2. Ability to stop API flow, also stop flow from scheduler
3. Add Find Usage for flows (within nested flows)
Done
8
votes
New fields' mapping editor
Enhance field mapping editor:
- add support for nested fields in the destination
- add an ability to modify attributes, such as JSON array, XML CDATA, XML ...
7
votes
Access Control for Shared Components in ETLWorks
ETLWorks would have the capability to lock a selection of formats, flows, and connections to particular user groups. This is because they can be repeatable shared ...
7
votes
Add retry functionality to agent schedules
ETLWorks host based schedules support: retries (1 - 5), delay between retries (1 - 10 minutes) and send failure email only if last retry failed. It would be really ...
7
votes
Excluding schedules or flows from the execution notification
I do like having the execution notification in the top right corner of the page, however there are some scheduled tasks which run so frequently that it would make the ...
7
votes
Provisioning from AWS and Azure Marketplaces
Add ability to provision Etlworks from AWS and Azure Marketplaces with Amazon and Microsoft billing
6
votes
Better ways to determine how flows, connections, etc. are interconnected
Right now when you have multiple flows that share connections or other flows, it's difficult to understand how making a change to one will possibly impact other ...
6
votes
Saved Views Based on Tags
We’d like to request “Saved Views” on ETLWorks pages that support tags- starting with Schedules, Flows, Connections, Agents, and Explorer pages.
- These views ...
6
votes
Filtering audit logs by time
Currently, audit logs can be filtered by the usable fields and a date range. Having a large amount of entry's, makes the log hard to use with these specifications, if ...
6
votes
Email on schedule failure only if last retry
I don't want emails for transient errors if a retry worked
Done
6
votes
Improved Filter/Search in Flow Statistics Page
It can be cumbersome to locate a specific job in the Flow Statistics page since it loads the jobs in the order in which it was run. Being able to search or filter ...
6
votes
IBM i Debezium Connector
Hello, would it be possible to add IBM i/AS400 Debezium Connector capabilities via this repo please?
https://github.com/jhc-systems/debezium-connector-ibmi
6
votes
Ability to schedule and execute flows by data integration agents running behind the firewall
Basically, add support for hybrid data integration when flows can be executed on data integration agents running behind the firewall.
6
votes
"Running Flow Tasks" in Statistics page to include agent tasks that are running
I'd like to be able to get a birds-eye view on all tasks (including both ETLWorks host and agent tasks) that are running. Currently, we can only get this through ...
Under consideration
6
votes
Auto-completion in JavaScript code editors
Add auto-completion in JavaScript code editors:
- {field names}
- functions
Planned
5
votes
Implement priority queue and throttling for user-defined APIs (listeners)
Scenario 1: upstream system slammed the system with thousands of API calls in a short span of time, eventually freezing and requiring a reboot to stop all jobs ...
5
votes
Add message and script to scheduler email
I want to add a message to the scheduler that contains all the handled flow exceptions.
Add a message field where we can append additional text to the default email ...
5
votes
Append Mode Option in Flows
Feature Name: Append Mode
Purpose: To provide a seamless way to append data to the destination without overwriting existing records.
Implementation Suggestion: A ...
5
votes
Statistics Executions: Sort and Filter Updates
On the statistics page, specifically the Executions area. The ability to:
1. Sort the results by the column headers. At least the started, ended, duration, and ...
5
votes
Interactive Performance recommandations
It would be a good idea to get performance recommandations based on one or more (nested) flows. Goal is decrease (processing time) of the flows. Maybe this can be ...
5
votes
Default tenant at login
Allow user to set default tenant (e.g., dev versus prod) when logging in. Currently defaults to no tenant.
5
votes
Repeat long running schedule notifications
When telling a schedule to alert you if it’s been running for more than X minutes, allow it to alert you every X minutes. Right now I can’t tell the difference ...
5
votes
Add exceptions/exclusions rules to schedules
Often times we need to apply schedules to not run at certain times for various reasons. Right now, we have to create multiple schedules. It would be nice to ...
5
votes
More Exporting Options for Logs / Flow Statistics
Currently the "statistics" module shows the daily execution statistics. However:
- There doesn't seem to be a way to filter results (say I wanted to see monthly ...
Done
5
votes
Search functionality / better documentation of flows
We use a good deal of javascript and in complex flows it can be really difficult to debug, esp. global and flow variables because there's no way to search (that I ...
5
votes
Add API Endpoints for Managing Individual Agent Flows
We’d like the ability to manage individual flows within an agent via the API, rather than having to submit the entire agent definition when making changes. ...
Under consideration
5
votes
Truly dynamic connection and format
Currently adding dynamic connection and format requires actually adding a named connection to the flow. It would be great if both could be truly dynamic, resolved at ...
4
votes
Ability to purge files within Server Storage connection after specified number of days
I would like to be able to delete files within a Server Storage connection after a specific number of days. This can be useful if I generate file logs into a specific ...
4
votes
A separate flow type, to stop other flows.
This feature being implemented as an actual flow, would allow to manage connectivity between flows, in more complex scenarios. Currently, the same behaviour is ...
Done
4
votes
Tenant name in Job failure notifications
can we add tenant name in failure email notification for an ETL job, this will help in the situations where there are multiple tenants.
Adding this will help to go ...
Done
4
votes
Direct URL to Specific Flow Execution within History
Currently, when investigating a specific execution in ETLWorks, users must manually locate it within the execution history page. This process is time-consuming, ...
4
votes
Multi select flows for bulk actions (delete, add tags, move to tenant) etc.
Would like to be able to select via checkbox or "all" results of a search and perform a bulk action on a set of flows, connections, users, etc.
4
votes
Connector for Google Analytics 4,
Since Google is discontinuing Universal Analytics and moving to only supporting Google Analytics 4,
4
votes
FLOWs visual based on tree hierarchy
When having multiple flows, it would be nice if you can collapse actions into the nested flows (like a tree).
Result then, only nested flows will be shown, until ...