Ideas for Etlworks Integrator as submitted to our Feature Upvote board. Ideas are ordered by 'new' and the 50 top matches are included. pr_vpumizna4jcgdwy 2025-05-18T12:19:03Z sug_wziffovx6perc0u 2026-03-30T22:54:56Z 2026-04-01T18:09:07Z If there are more than 20 steps under File to Database ETL Flow, then if we want to make duplicate the 1st step, we need to scroll entirely down in order to scroll right side, earlier versions we used to see scroll bar irrespective of how many steps we have in one ETL flow. this will help us not to scroll till end to scroll to right side using scroll bar. sug_ec4o9oejyqvecdx 2026-04-01T18:06:10Z 2026-04-01T21:48:09Z Add support for API keys with configurable expiration (for example, 7 or 30 days) to enable temporary user access without requiring token generation or refresh logic. sug_ozitqjlg85cdeei 2026-03-24T18:17:01Z 2026-04-02T16:41:20Z There will be some scenarios, where we need to use one ETL flow that was created on one Tenant in another Tenant. Currently, if we want to use same ETL flow that was created in one Tenant, we need to create same in another Tenant and use. If we have this option, then we can reuse the same ETL Flow across multiple Tenants. sug_hbaob0z63mljclg 2026-02-11T21:44:47Z 2026-02-15T15:49:40Z I really appreciate the hierarchical structure within the Flow UI, it makes complex pipelines much easier to manage. I would like to request an enhancement to allow keyword search within the contents of a Flow, not just by step titles. For example, the global search allows searching for terms like "baseFileName" and returns all assets referencing it. It would be very helpful if the Flow-level search could have the ability to also scan within the flow. While having that visibility across all assets is ideal, being able to search keywords directly within an individual Flow would still be extremely valuable. Being able to quickly locate internal references inside large nested flows would significantly improve maintainability and troubleshooting efficiency. Thanks! sug_grxd0rfqe4uq5oa 2026-02-11T07:32:29Z 2026-02-15T15:50:16Z Add a “Recently Viewed” sidebar, similar to “Recent Commands,” to the Connections, Flows, Agents, and Schedules pages. For example, if I am on the Flows page and I open: - Flow ABC - Flow DEF - Flow GHI Then a “Recently Viewed Flows” panel should display those items in most-recent-first order, allowing me to click directly back into any of them without searching again. The same behavior should apply to Connections, Agents, and Schedules. This would reduce navigation friction and improve efficiency in large environments where users repeatedly move between the same entities during troubleshooting or development. sug_qmrscve0ytbagsw 2026-02-08T18:26:17Z 2026-02-15T15:50:40Z We currently upgrade ETLWorks agents via the UI, but we manage hundreds of agents and need a more scalable, automatable approach. Would you consider adding an API endpoint to trigger agent upgrades (optionally to a target version), with asynchronous job status and basic pre-checks? This would allow us to automate upgrades during maintenance windows, improve auditability, and reduce manual operational work. sug_kczbdbfsaa7nwfe 2026-01-23T16:12:02Z 2026-03-24T17:21:17Z In our jobs, we found that when one or more provided lookup fields in the Parameter Tab does not exist in source table, Etlworks will ignore the column and continue the job without any error in the log. This ignorance of the invalid column cased the data discrepancy between source and destination especially when lookup fields has multiple values. We would like ETLWorks to raise the error and stop the job instead of continue processing the data, this will be much better than giving the wrong result without any indication. sug_1lgp98iukg50xi7 2025-12-30T15:04:06Z 2025-12-31T14:43:48Z Either allow the Execute SQL flow to work against a NoSQL connection like MongoDB or create a new flow for Execute NoSQL. This would allow more complex operations in Mongo like those that are currently possible in Snowflake and other SQL databases. sug_wikft0tv5ndkqay 2025-11-07T08:36:09Z 2025-11-11T21:20:25Z When configuring an ETL schedule we mostly enable the same options over and over again. e.g. enable send mail on failure, enable only on last retry, add Delay between retries 10min, add number of retries and always add the same e-mail address. Could it be a feature to configure a template for this with default values? Maybe the default emailadress for notifications should be somewhere else as a general setting and could be overwritten when needed in the schedule? sug_fd46wb8ulhei7wl 2025-10-30T20:48:49Z 2025-11-03T20:18:43Z It would be very helpful if the ETLWorks Admin Settings page displayed node system details such as CPU cores, total RAM, and disk capacity. This visibility would assist with capacity planning and troubleshooting, and reduce support tickets needed to confirm environment specs. Other ETL platforms include this information directly in their admin consoles. sug_35cl28dzjvuldsw 2025-09-24T22:27:33Z 2025-09-25T17:50:33Z Problem In Explorer, all connections are currently displayed in a single list. When managing many connections, it becomes difficult to quickly find the one you need, especially when names are similar or when working across multiple connection types (for example, SQL Server, S3, FTP). Manual scrolling or typing partial names is inefficient. Proposed Solution Add a filter option in Explorer to show only connections of a given connection type. For example: - Filter to show only SQL Server connections - Filter to show only S3 connections - Filter to show only FTP connections This could be implemented as a dropdown (like Tags is) or advanced search criteria, integrated with the existing Explorer search box. Benefits - Faster navigation and reduced time spent scrolling through long connection lists. - Easier onboarding for new team members, since they can quickly isolate the type of connection they’re working with. sug_3szc654698xzhmw 2025-09-22T22:58:59Z 2025-09-26T18:12:25Z Currently, ETLWorks executes the same Preprocessor and Postprocessor scripts for both the authentication (ie OAuth2/token) call and the main HTTP request. This forces developers to add conditional branching logic in the scripts to differentiate between token responses and actual API responses. It would be helpful to add an option to scope processors more granularly, for example: - Run Preprocessor/Postprocessor for authentication only - Run Preprocessor/Postprocessor for main request only - Run Preprocessor/Postprocessor for both (default, preserves current behavior) Alternatively, providing a runtime context flag (e.g., context.isAuthenticationPhase()) that can be checked in scripts would also address this issue. Benefits: - Simplifies scripting logic; no need to manually parse/branch based on response shape. - Reduces risk of errors when authentication and main requests return similar JSON structures. - Improves clarity for developers working with complex integrations. sug_yhzplrknmjk9tag 2025-09-22T22:28:32Z 2025-11-11T21:20:59Z Requests: Phase 1: Add environment-scoped variables editable only by Super Admins; referenceable via (like `{VAR_NAME}`) in flows, connections, etc Use cases: rotating API keys; base URLs. Phase 2 (more of a nice-to-have): Make variables non-viewable to non–Super Admins; any props.get on a secret returns a masked or encrypted value Use cases: Meet compliance requirements by ensuring operators (like customer support) cannot view sensitive values. Suggestion for UI placement: Implement in the SuperAdmin tenant under Settings → Customization; similar to existing Super Admin dropdown fields sug_eu910ugdvir26o2 2025-09-22T21:04:37Z 2025-09-23T13:05:41Z We’d like to request an enhancement to how ETLWorks reports flow execution statistics. - Nested flows: Nested flows shows 0s duration in the UI, even though its child flows take several minutes. It would be more accurate to an operator if the parent flow displayed its start and finish times spanning all children, with total runtime rolled up. - Loops: Loop numbering in execution history can appear out of order (e.g., loop “2” listed before loop “1”), which makes troubleshooting harder. - Hierarchy view: Adding an expand/collapse view for nested flows (default expanded) would help operators quickly see child flow details and durations without losing context. These enhancements would make the Statistics view more accurate and intuitive for operators and support teams, reducing confusion and speeding up troubleshooting. sug_vrvxl0tp8csv10q 2025-09-08T17:56:40Z 2025-09-26T18:27:25Z On Sundays our main transactional system reboots. We'd like flexibility to in the flow scheduling to configure something like: On Sundays, 2-3 AM. sug_q2pnqcwoxcbtffr 2025-08-05T16:49:21Z 2025-08-05T16:57:52Z Sometimes I have a lot of files that I want to delete (in a timespan) within a Local Storage connection, but because I have to delete one file at a time, it is tedious and time consuming. It would be great if I could just hold shift to bulk select a group of files, and then be able to click the trash button after that. The other indirect benefit is that this can help reduce the amount of storage space overall in the instance by making this an easier process to maintain. sug_q4gwi3mzcf2rh18 2025-07-16T05:36:53Z 2025-09-22T19:00:59Z Possibility to click on a column in statistics and get the corresponding values in the rest of the statistic (Instead of choosing the date from the Datelist). If you click on the errors in the Pie Chart you immediately see the corresponding flows with errors (Instead of manually setting the Filter in the Execution list) sug_pw75cv1zazupwyi 2025-05-27T19:46:12Z 2025-05-28T13:54:48Z We’d like the ability to manage individual flows within an agent via the API, rather than having to submit the entire agent definition when making changes. Currently, the Edit Agent API requires re-submitting the full agent object, including all flows, even when we only need to modify one. This introduces unnecessary complexity when automating changes at scale. Would you consider exposing a set of flow-level endpoints, such as: GET /agents/{agentId}/agentFlows/{agentFlowId} POST /agents/{agentId}/agentFlows PATCH /agents/{agentId}/agentFlows/{agentFlowId} DELETE /agents/{agentId}/agentFlows/{agentFlowId} Benefits of this enhancement: - Reduces the risk of unintentionally overwriting unrelated flows during updates - Simplifies automation logic by allowing more granular control - Improves traceability and auditability during restores and deployments - Aligns better with how Host schedules are managed today sug_ngbrnosfp0xa8mv 2025-05-21T21:54:15Z 2025-05-22T14:15:27Z I love the global search feature for searching for a keyword, however sometimes it might not be that useful if I have many pipelines that have that keyword. I have two unique suggestions that can make us faster to find a keyword: 1) The global search has a limit of 20, there's no way to scroll down if there are more 2) Restrict searching a keyword for a specific nested flow that I'm interested in. For example, let's say I wanted to search the keyword within a nested flow's 20 child flows (including all the different levels). sug_6vdysusmp4wmcgg 2025-05-19T20:30:53Z 2025-09-22T18:57:56Z I’d like to expose the ID of the currently executing schedule (if the flow is triggered by a schedule) as a global-defined variable (like the ones listed in https://support.etlworks.com/hc/en-us/articles/360016579573-Default-variables). Use case: This would make it easier to build flows that can adjust behavior dynamically based on the schedule that triggered them. It would also reduce the need for hardcoding or manually simulating schedule behavior during testing and development. In particular, when a flow is manipulating schedule-related data, and there’s no way to programmatically exclude the schedule that triggered it, there’s a risk of unintentionally altering the very schedule that’s running. While this could be avoided by hardcoding the schedule ID, that approach doesn’t scale well or work cleanly across environments. Having access to the schedule ID at runtime would improve safety, flexibility, and portability of schedule-based flows. sug_shoo9xp9ediqcp2 2025-04-24T04:42:30Z 2025-09-22T19:01:25Z Could we be able to filter for Listeners on the Schedules page? It would make it easier to see how many we have available. sug_cugmxjbuhwvjslj 2025-04-23T17:08:32Z 2025-12-17T21:54:52Z We’d like to request “Saved Views” on ETLWorks pages that support tags- starting with Schedules, Flows, Connections, Agents, and Explorer pages. - These views would filter assets by tag, with simple match patterns (e.g., tag contains or does not contain “plc_”). This logic already exists when filtering by a single tag- this would extend it to show assets containing a group of related tags. - Views could be global (shared) and/or personal (user-specific). This would make it much easier for teams to find relevant assets, especially when focused on specific suites or tools meant for specific teams. It would be just as helpful, if not more, than previously requested folder-based tag organization. Long-term, views on assets could even support permission-based access. sug_iuuifjpatzlk15q 2025-04-18T20:48:47Z 2025-04-18T22:07:13Z I'd like to be able to get a birds-eye view on all tasks (including both ETLWorks host and agent tasks) that are running. Currently, we can only get this through ETLWorks host tasks. Otherwise, the only way we can tell that an agent task is taking a long time if we set the expected execution time notification for the email. But this isn't at a macro level. This is just valuable observability information to have in general. sug_h2d4u1egutxh2uv 2025-03-12T18:26:03Z 2025-09-22T18:57:10Z Currently, when investigating a specific execution in ETLWorks, users must manually locate it within the execution history page. This process is time-consuming, especially when handling numerous executions or collaborating across teams. Proposed Solution: Modify the existing execution history URL format to include an execution_id query parameter that allows direct navigation to a specific execution. Current URL Format: https://opengov.etlworks.com/#/app/schedules/243/statistics Proposed URL Format: https://opengov.etlworks.com/#/app/schedules/243/statistics?execution_id=1234 By appending `?execution_id=<value>`, users can share and open URLs that directly reference a specific execution within the execution history. sug_gvhy7nfdvqoqbmg 2025-03-04T14:46:53Z 2025-12-17T21:58:15Z ETLWorks would have the capability to lock a selection of formats, flows, and connections to particular user groups. This is because they can be repeatable shared components, so if someone from the wrong team touches them, it can be detrimental sug_8jtm4inoochymmz 2025-03-03T14:27:18Z 2025-03-03T14:50:56Z Whenever a File is loaded into a database we have the possibility to delete the file after successful processing. (Delete loaded source files ) It would be helpful, if we could move the file instead to a specific subfolder (move loaded source files) & text field with name of subfolder Right now we have to 1. Step: Flow Feile to Database 2. Step: Move File 3. Step: Kombine Step 1 & 2 in nested flow 4. Step: Loop with wildcard. This would be much easier then. sug_te5315uo5nuiavo 2025-02-27T19:22:54Z 2025-03-07T21:29:22Z Click through flows to child items (connections, flows, parameters) in flows. If we're in a flow, we always have to search for the flow which we allready see in our window. It would be very helpfull if we could (for instance) ctrl click on the flow and that it will open the flow / connection / or what ever type is in your flow. sug_harfgt0ly9vzwpg 2025-02-04T17:53:28Z 2025-02-04T20:07:25Z We have a shared pipeline with a hierarchy of nested flows. When an error occurs in a deeply nested flow, our support team needs to determine its execution context (parent flow, grandparent flow, etc.) for efficient troubleshooting. I propose the following solutions but am open to alternative implementations from ETLWorks: - Option 1: Retrieve the entire hierarchy at the start of the pipeline and store it as a reference. When an error occurs, use the failing flow's name to look up its position in the stored hierarchy and determine its parent-child relationships. The execution context can then be formatted as a string variable and sent to an observability tool. - Option 2: Provide a function that dynamically returns the current execution context, including parent flows, as a string. This feature would significantly improve debugging. This request follows up on support request #9799. Please let me know if more details are needed. sug_r71tnu3y4fpeuqv 2025-01-28T19:51:43Z 2025-02-22T20:32:12Z 1. If configured throw an error if there is column in the source which does not exist in mapping 2. If configured ignore all columns in the source which are not in mapping (automatically disable these columns) sug_yunaconsc1bbahy 2025-01-28T19:43:12Z 2025-02-08T18:57:04Z Currently adding dynamic connection and format requires actually adding a named connection to the flow. It would be great if both could be truly dynamic, resolved at runtime. sug_vmxbif6bdhraocz 2025-01-26T21:21:05Z 2026-03-11T14:31:25Z Currently each agents needs to be updated individually. From our perspective, for bulk updating, I would say it would be managed via the Agents page itself. - The ability to filter on that page based on Version (less than version X) - The ability to filter on that page based on Status too (if someone is offline, we probably dont want to send/queue the update as there could be some other underlying issue - On that page, if there were perhaps checkboxes to select which agents to send the update command to (with an option up top for Select all, which would only select the ones that have been filtered out/being viewed in the UI at the time) - If there were a way to have some sort of logging to show both success and errors in updating. Errors for the obvious reason that they didnt update, and successes are for the rare cases a customer might complain and blame something on the update itself, so having a log of when the update took place could be nice. sug_nfoirn5bh4c3ard 2025-01-17T20:27:31Z 2026-01-19T21:12:15Z can we add tenant name in failure email notification for an ETL job, this will help in the situations where there are multiple tenants. Adding this will help to go to that tenant and resolve the job issue. sug_wz8vv5anbebkhfs 2025-01-16T21:05:08Z 2025-09-26T18:07:11Z Feature Name: Append Mode Purpose: To provide a seamless way to append data to the destination without overwriting existing records. Implementation Suggestion: A simple checkbox in the flow configuration labeled “Append Mode.” When selected, the flow will append new records to the destination without deleting or overwriting existing data. When unselected, the existing behavior (replace or overwrite) will remain as is. Use Case Example: If the source contains 1 record, running the flow three times should result in 3 records in the destination, with each execution appending data instead of replacing it. sug_jgqebtvx91qr6x6 2025-01-14T18:01:51Z 2025-01-14T18:15:59Z In the agent flow, when going to the "Configure flow notifications", when I enter an amount of minutes for the "Expected flow execution time", it currently does not send an email when the execution exceeds that amount of time. Can this feature please be implemented? It would be very useful for our support teams to know which agent flows have been stalling. sug_05f1m5ljynb9rfh 2025-01-13T21:39:09Z 2026-01-19T21:01:36Z It will be great if there be ability to order the flows in agent\schedule panel sug_w8vtz3qyhrbg2ld 2024-12-06T13:55:37Z 2025-09-22T18:58:54Z Currently, audit logs can be filtered by the usable fields and a date range. Having a large amount of entry's, makes the log hard to use with these specifications, if youre looking for something that happened at a specific time, rather than a specific action. The possibility to also filter by time would solve this problem, as you could easily narrow down the searchspace to a usable amount. sug_pug5qx2se1cmvnm 2024-11-09T05:24:13Z 2024-11-29T13:57:03Z ETLWorks host based schedules support: retries (1 - 5), delay between retries (1 - 10 minutes) and send failure email only if last retry failed. It would be really helpful if Agent schedules supported the same functionality. Sometimes there can be a transient db connection errors with a customer's db or server, or some other issue that can easily be resolved with retries. The ability to only email if the last retry failed helps cut down on extra emails. sug_yooghvkd1hpqsgh 2024-10-25T15:54:17Z 2025-09-22T19:02:20Z It can be cumbersome to locate a specific job in the Flow Statistics page since it loads the jobs in the order in which it was run. Being able to search or filter based on information that is available in the Records Metrics or File Metrics page (i.e. File Name) would improve troubleshooting capabilities by making it easier to find specific instances of a run of the flow. sug_fsxlt2ckrkfm6ee 2024-10-21T15:23:37Z 2026-03-11T14:32:58Z Would like to be able to select via checkbox or "all" results of a search and perform a bulk action on a set of flows, connections, users, etc. sug_thfbccnjvbbrjrb 2024-10-07T16:49:43Z 2025-09-22T19:01:52Z On the statistics page, specifically the Executions area. The ability to: 1. Sort the results by the column headers. At least the started, ended, duration, and record columns 2. Add to the filter rules: a. Number of records. optimally a greater than, less than, or equal to. i.e. I'd love to filter on just flows with a 0 or greater than 100k. b. Duration. i.e. Where duration > 1 hour or start >1am and end < 5am Alternatively, being able to export the day, from that page, as a CSV I could do these things as well. Love to be able to just stay in the platform, but an export would work. And I know there is a potential to do something with the API, but hoping it's possible directly in app. sug_3i5sixfknpwcyqo 2024-08-30T17:01:14Z 2024-12-11T15:48:10Z I would like to be able to delete files within a Server Storage connection after a specific number of days. This can be useful if I generate file logs into a specific Server Storage connection and I don't want to have too much clutter. sug_yt3f70qqccgwlxi 2024-08-28T14:51:20Z 2025-12-17T21:52:48Z HI sometimes we want to filter on flows which have 1 specific tag and onother specific tag. Now it isn't possible to do that. Maybe it is an idea to make this as a new feature. Maybe it is also a good idea that you can save your own filters as a bookmark or something. sug_myo7l4g9t3bsqif 2024-08-14T16:28:56Z 2024-08-30T17:25:05Z Currently right now when stopping etlworks or performing an upgrade, we manually have to stop everything or when etlworks starts back up, the flows are still showing as running. It would hugely beneficial for the startup process to reset the status of the flow so that schedules automatically resume. sug_jv7xswnwbzdak3j 2024-08-06T16:53:42Z 2026-03-25T18:53:59Z It would be nice if there was a Recycle Bin for flows (or other assets), and it could have a time limit (like 30 days). This way, if one of our users accidentally deletes something, it'll be really easy to restore. sug_vwf0grt80efxe2n 2024-08-06T03:51:30Z 2026-01-19T21:03:21Z I do like having the execution notification in the top right corner of the page, however there are some scheduled tasks which run so frequently that it would make the feature useless. Thus, if there was a way to opt out of the notification for select flows/tasks, that would be nice to have. This is preferable to disabling execution notifications entirely. sug_nxngzu5ff01iptd 2024-07-11T09:34:51Z 2024-07-12T13:14:42Z A this moment we have to interpretate the flowstep running time by calculating Start / Stop . It should be handy if you could see this at once. We use this qiut often. sug_xyeidczvkkfb8t2 2024-06-19T10:35:47Z 2025-08-20T15:29:47Z It would be a good idea to get performance recommandations based on one or more (nested) flows. Goal is decrease (processing time) of the flows. Maybe this can be accomplished by analysing Amount of Data (columns/rows) vs settings per FlowType vs Result (processing time) sug_brwgrbilz2jry1x 2024-05-21T07:14:58Z 2025-05-05T13:46:43Z Hi, In the case of CDC flows with frequent schedules it often happens that there are no new records. Current behavior: an empty file is created and copied to AWS S3 and then to Snow in our case ( MS SQL to Snow via S3 flow ). It consumes time and resources. Copying from S3 to Snow ( even an empty file ) consumes most of the time for the "List external files" operation. Proposition: If there are no new records, don't create a file. sug_frk5bsehbqwyax1 2024-04-10T12:43:36Z 2025-11-03T20:15:51Z As a user, creating database lookups requires the user to know code in the field value function to perform database lookups. This is even more complicated when doing CDC tooling. It would be nice to have a feature on each field in a transformation that would launch a popup allowing the user to configure the following: - Database connection - Table - Column - SQL query to inject parameters This UI would essentially wire up the lookup behind the scenes and allow the user to quickly configure these things, streamling creating flows. sug_xvsj5s3bccncqss 2024-02-23T21:13:15Z 2024-02-26T17:15:52Z Please add support for folders and subfolders in zip/unzip flow so we can specify path in FROM and TO field both directly in the field and through parameter in the connection.