{"id":23480,"date":"2023-08-17T13:15:59","date_gmt":"2023-08-17T16:15:59","guid":{"rendered":"https:\/\/ee02395c61.nxcli.io\/insights\/apache-nifi\/"},"modified":"2024-04-04T12:04:58","modified_gmt":"2024-04-04T15:04:58","slug":"apache-nifi-aws-cognito-data","status":"publish","type":"insights","link":"https:\/\/elogroup.com\/en\/insights\/apache-nifi-aws-cognito-data\/","title":{"rendered":"Apache NiFi and AWS Cognito in the context of a data-driven culture"},"content":{"rendered":"\t\t<div data-elementor-type=\"wp-post\" data-elementor-id=\"23480\" class=\"elementor elementor-23480 elementor-18010\" data-elementor-post-type=\"insights\">\n\t\t\t\t\t\t<section data-particle_enable=\"false\" data-particle-mobile-disabled=\"false\" class=\"elementor-section elementor-top-section elementor-element elementor-element-f0783ea elementor-section-boxed elementor-section-height-default elementor-section-height-default\" data-id=\"f0783ea\" data-element_type=\"section\" data-e-type=\"section\">\n\t\t\t\t\t\t<div class=\"elementor-container elementor-column-gap-default\">\n\t\t\t\t\t<div class=\"elementor-column elementor-col-100 elementor-top-column elementor-element elementor-element-59e2c41\" data-id=\"59e2c41\" data-element_type=\"column\" data-e-type=\"column\">\n\t\t\t<div class=\"elementor-widget-wrap elementor-element-populated\">\n\t\t\t\t\t\t<div class=\"elementor-element elementor-element-4293380 elementor-widget elementor-widget-text-editor\" data-id=\"4293380\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"TextRun SCXW169143497 BCX0\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW169143497 BCX0\">By<\/span><span class=\"NormalTextRun SCXW169143497 BCX0\"> Felipe <\/span><span class=\"NormalTextRun SCXW169143497 BCX0\">Gochi<\/span><span class=\"NormalTextRun SCXW169143497 BCX0\">, in collaboration with<\/span><span class=\"NormalTextRun SCXW169143497 BCX0\">\u00a0Fabio <\/span><span class=\"NormalTextRun SCXW169143497 BCX0\">Catein<\/span> <span class=\"NormalTextRun SCXW169143497 BCX0\">and<\/span><span class=\"NormalTextRun SCXW169143497 BCX0\"> Rodrigo Bassani<\/span><\/span><span class=\"EOP SCXW169143497 BCX0\" data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3ca33e9 elementor-widget-divider--view-line elementor-widget elementor-widget-divider\" data-id=\"3ca33e9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-79e19a3 elementor-widget elementor-widget-text-editor\" data-id=\"79e19a3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<ul><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><i><span data-contrast=\"auto\">Apache NiFi is an open-source data processing and orchestration tool designed to ease information\u2019s movement between different systems in real time.\u00a0<\/span><\/i><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"2\" data-aria-level=\"1\"><i><span data-contrast=\"auto\">Because it is highly scalable and fault-tolerant, it is well suited to dealing with large volumes of data in distributed environments.\u00a0<\/span><\/i><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"1\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"3\" data-aria-level=\"1\"><i><span data-contrast=\"auto\">These characteristics make Apache NiFi a powerful enabler from a technology and business perspective, unlocking important value for organizations.<\/span><\/i><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8dbf8c6 elementor-widget-divider--view-line elementor-widget elementor-widget-divider\" data-id=\"8dbf8c6\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-197640a elementor-widget elementor-widget-text-editor\" data-id=\"197640a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW21344415 BCX0\">Within<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> a <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">context<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> in <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">which<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">organizations<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> are <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">demanding<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">increasingly<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">intelligen<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">ce<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> in <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">capturing<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">and<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">processing<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> data, <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">in <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">order<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">to<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">cultivate<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> a <span style=\"text-decoration: underline;\"><span style=\"color: #0000ff; text-decoration: underline;\"><a style=\"color: #0000ff; text-decoration: underline;\" href=\"https:\/\/elogroup.com\/en\/insights\/data-centric-artificial-intelligence\/\">data-<\/a><\/span><\/span><\/span><span style=\"text-decoration: underline;\"><span class=\"NormalTextRun SCXW21344415 BCX0\" style=\"color: #0000ff; text-decoration: underline;\">driven<\/span><\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">culture<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">, Apache <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">NiFi<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">presents<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">itself<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> as a <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">robust<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">and<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">flexible<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">solution<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">. <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">This<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> tool <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">has<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">the<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">potential<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">to<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> improve <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">the<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">way<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">organizations<\/span> <span class=\"NormalTextRun ContextualSpellingAndGrammarErrorV2Themed SCXW21344415 BCX0\">handles<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> data, as <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">well<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> as <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">guaranteeing<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">greater<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">security<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">to<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> it<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">. In <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">this<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">article<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">, <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">we<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">wi<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">ll<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> explore in <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">detail<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">how<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">this<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> tool <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">works<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">and<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">how<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> it <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">can<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">generate<\/span> <span class=\"NormalTextRun SCXW21344415 BCX0\">value<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> for <\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">the<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\"> business<\/span><span class=\"NormalTextRun SCXW21344415 BCX0\">.\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-88acc50 elementor-widget elementor-widget-heading\" data-id=\"88acc50\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">What is Apache NiFi? <\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-18eb444 elementor-widget elementor-widget-text-editor\" data-id=\"18eb444\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW131370359 BCX0\">Apache <\/span><span class=\"NormalTextRun SCXW131370359 BCX0\">NiFi<\/span><span class=\"NormalTextRun SCXW131370359 BCX0\"> is an open-source data processing and orchestration tool designed to <\/span><span class=\"NormalTextRun SCXW131370359 BCX0\">help<\/span><span class=\"NormalTextRun SCXW131370359 BCX0\"> the movement of information between different systems in real time. It provides an intuitive graphical interface for creating, <\/span><span class=\"NormalTextRun SCXW131370359 BCX0\">managing<\/span><span class=\"NormalTextRun SCXW131370359 BCX0\"> and <\/span><span class=\"NormalTextRun SCXW131370359 BCX0\">monitoring<\/span><span class=\"NormalTextRun SCXW131370359 BCX0\"> these flows. Apache <\/span><span class=\"NormalTextRun SCXW131370359 BCX0\">NiFi<\/span><span class=\"NormalTextRun SCXW131370359 BCX0\"> is highly scalable and fault-tolerant, which makes it suitable for handling large volumes of data in distributed environments. It allows users to create the so-called \u201cdata flows\u201d by simply and intuitively dragging and dropping pre-built components, \u201cprocessors<\/span><span class=\"NormalTextRun SCXW131370359 BCX0\">\u201d,<\/span><span class=\"NormalTextRun SCXW131370359 BCX0\"> onto its graphical interface.\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-5906ca9 elementor-widget elementor-widget-heading\" data-id=\"5906ca9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">How does the Apache NiFi architecture work? <\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8857fa9 elementor-widget elementor-widget-image\" data-id=\"8857fa9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img fetchpriority=\"high\" decoding=\"async\" width=\"800\" height=\"474\" src=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-27-1-1024x607.png\" class=\"attachment-large size-large wp-image-21018\" alt=\"\" srcset=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-27-1-1024x607.png 1024w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-27-1-300x178.png 300w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-27-1-768x455.png 768w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-27-1.png 1302w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ededae5 elementor-widget elementor-widget-text-editor\" data-id=\"ededae5\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">The Apache NiFi architecture runs on a JVM (Java Virtual Machine) hosted on the Operating System chosen by the user based on their specific needs.\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><ul><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Web Server:<\/strong> is the repository of NiFi command and control APIs in HTTP;\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"2\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Flow Controller:<\/strong> from these APIs, it provides threads or parallel processing and manages the scheduling of extensions so that they are executed;\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"3\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Extension\/Processor:<\/strong> these are mechanisms and operations that can be executed within the JVM, generating data files;\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"4\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>FlowFile Repository:<\/strong> stores and checks the execution and status of this data; and<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"2\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"5\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Content Repository:<\/strong> handles storing the contents of this data. There can be more than one repository, where data is physically fractionated to make better use of space and storage.<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><\/ul><p><span data-contrast=\"auto\">\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">NiFi processors offer a wide range of functionalities, such as data transformation, enrichment, filtering, routing and integration with external systems, among others. These processors can be configured and connected to create complex flows, allowing data to be ingested, processed and delivered in real time.<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-277725a elementor-widget elementor-widget-image\" data-id=\"277725a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"800\" height=\"450\" src=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-22-1024x576.png\" class=\"attachment-large size-large wp-image-21013\" alt=\"\" srcset=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-22-1024x576.png 1024w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-22-300x169.png 300w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-22-768x432.png 768w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-22-1536x864.png 1536w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-22.png 1920w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-d4d24d8 elementor-widget elementor-widget-text-editor\" data-id=\"d4d24d8\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">Apache NiFi is widely used in many areas, like in data analysis, Internet of Things (IoT), log data processing, systems integration and data migration. Its flexibility, scalability and advanced features make it a powerful tool for dealing with the flow of data in complex and constantly evolving environments. Also, since it is Apache Software Foundation\u2019s open-source project, NiFi has an active community of developers who contribute with improvements and added features.\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-3b9695d elementor-widget elementor-widget-heading\" data-id=\"3b9695d\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Generating value with Apache NiFi <\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ed660ad elementor-widget elementor-widget-image\" data-id=\"ed660ad\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" width=\"764\" height=\"1024\" src=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-28-764x1024.png\" class=\"attachment-large size-large wp-image-21014\" alt=\"\" srcset=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-28-764x1024.png 764w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-28-224x300.png 224w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-28-768x1030.png 768w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-28-1145x1536.png 1145w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-28.png 1490w\" sizes=\"(max-width: 764px) 100vw, 764px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-2540658 elementor-widget elementor-widget-text-editor\" data-id=\"2540658\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">In this subject, an extremely important topic is the CAF (Cloud Adoption Framework). It is a complete guide to the efficient and secure adoption of cloud computing, covering technical, organizational and operational aspects. With a focus on governance, necessary skills, architecture, security, operations and cost optimization, the CAF helps companies align their goals with a strategy that promotes collaboration and innovation. The main cloud platforms offer services focused on CAF, such as the AWS Cloud Adoption Framework.\u00a0\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">The image above shows how we can achieve this goal by adopting Apache NiFi in conjunction with Kubernetes to execute CAF frames in specific cases (remembering that the adoption of these tools must consider the particularities of each project, as explained in our <\/span>article on <span style=\"text-decoration: underline;\"><span style=\"color: #0000ff;\"><a style=\"color: #0000ff; text-decoration: underline;\" href=\"https:\/\/elogroup.com\/en\/insights\/kubernetes-ally-cloud-journey\/\">Kubernetes<\/a><\/span><\/span><span data-contrast=\"auto\">).\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">Considering the characteristics of Apache NiFi, we can infer different capabilities that make it an enabling tool. Thus, we find different categories described in the Tech Enablers tab:<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><ul><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"3\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\">Deployment using container and virtualization technologies, such as Docker and Kubernetes;\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"3\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"2\" data-aria-level=\"1\"><span data-contrast=\"auto\">Devsecops controlled by the deployment environment easily integrated into different types of context;\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"3\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"3\" data-aria-level=\"1\"><span data-contrast=\"auto\">With different configuration modes and utilization modules, NiFi can be a crucial tool for integrations between local and remote networks (hybrid integration);\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"3\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"4\" data-aria-level=\"1\"><span data-contrast=\"auto\">It can be used as a SAAS with different types of operations; and<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"3\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"5\" data-aria-level=\"1\"><span data-contrast=\"auto\">Possibility of an event-based architecture.\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><\/ul><p><span data-contrast=\"auto\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">As a technological enabler for different purposes, its robustness combined with its flexibility also allows Business Drivers to be activated in the best way. Let\u2019s explore how Apache NiFi can become an enabler and generate value for the business.<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-69712f7 elementor-widget elementor-widget-heading\" data-id=\"69712f7\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Data ingestion platform <\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-85a7600 elementor-widget elementor-widget-text-editor\" data-id=\"85a7600\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span class=\"NormalTextRun SCXW10161394 BCX0\">Data <\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">ingestion<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">is<\/span><span class=\"NormalTextRun SCXW10161394 BCX0\"> a <\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">term<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">that<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">refers<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">to<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">the<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">process<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">of<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">collecting<\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">, <\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">importing<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">and<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">storing<\/span><span class=\"NormalTextRun SCXW10161394 BCX0\"> data in a system <\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">or<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">platform<\/span><span class=\"NormalTextRun SCXW10161394 BCX0\"> for later <\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">processing<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">and<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">analysis<\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">. It <\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">is<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">the<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">first<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">stage<\/span><span class=\"NormalTextRun SCXW10161394 BCX0\"> in <\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">the<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">flow<\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">, <\/span><span class=\"NormalTextRun SCXW10161394 BCX0\">where<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">raw<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">information<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">is<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">captured<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">from<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">various<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">sources<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">and<\/span> <span class=\"NormalTextRun SCXW10161394 BCX0\">prepared<\/span><span class=\"NormalTextRun SCXW10161394 BCX0\"> for later use.<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-19d464b elementor-widget elementor-widget-heading\" data-id=\"19d464b\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Consuming from multiple sources <\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9b78774 elementor-widget elementor-widget-text-editor\" data-id=\"9b78774\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">In a high-volume environment, the data ingestion process needs to deal with large volumes of data and ensure that it is captured, processed and stored in an efficient and scalable manner.\u00a0\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">There are some important aspects to consider in this process:<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><ul><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"5\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Diversity of sources:<\/strong> in a distributed environment, data can come from many different sources, such as databases, legacy systems, IoT devices, cloud services, etc.;<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"5\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"2\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Volume and speed:<\/strong> in a distributed environment, data can be generated in large volumes and at high speed. Dealing with the ingestion of substantial amounts of data requires a robust and scalable infrastructure to ensure that all data is captured and processed without loss or significant delays;<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"5\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"3\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Scalability:<\/strong> to deal with high volumetrics, the ingestion system needs to be able to scale, which means add more resources as demand increases. This can be achieved through a distributed architecture, where several nodes or servers work together to process the data in a parallelized and scalable way;<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"5\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"4\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Load distribution:<\/strong> distributing the workload between the different nodes in the system is essential to avoid bottlenecks and guarantee efficient processing. This can be done using techniques like data fractionation, intelligent routing and load balancing to ensure that each node is being utilized in a balanced way;<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"5\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"5\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Real-time processing:<\/strong> in a high-distribution environment, it is common for data to be generated and transmitted in real time. Therefore, ingestion ability must be designed to cope with the speed of data flow, allowing for real-time processing and storage without significant delays. This involves implementing real-time event processing systems and adopting technologies that support low latency; and<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"5\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"6\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Monitoring and management:<\/strong> in a high volume and distribution environment, it is essential to have comprehensive monitoring of the ingestion process. This involves tracking the ingestion rate, the performance of the processing nodes, detecting bottlenecks and being able to adjust resources as necessary. In addition, it is important to have centralized management tools and mechanisms to ease system administration and troubleshooting.\u00a0<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559737&quot;:-20,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><\/ul><p><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559685&quot;:0,&quot;335559737&quot;:-20,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">All these requirements for data ingestion in a high-volume environment with various sources demand a more refined control of the costs of supporting this architecture. We must, therefore, pay attention to some important aspects to reduce costs and optimize performance, such as:<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559685&quot;:0,&quot;335559737&quot;:-20,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><ul><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"6\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"7\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Use of shared resources:<\/strong> one of the main advantages of distributed computing is the ability to share resources between different workloads. Instead of having dedicated servers for each application, it is possible to use a shared infrastructure, like a cluster of servers, where different tasks and processes can be carried out. This allows for better utilization of available resources, reducing the need for dedicated infrastructure for each application;<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559737&quot;:-20,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"6\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"8\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Elasticity and scalability on demand:<\/strong> a well-designed distributed architecture allows resources to be scaled according to demand. With elasticity and scalability on demand, resources can be added or removed as needed. This makes it possible to adjust processing ability according to real needs, avoiding excessive costs with idle resources; and<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559737&quot;:-20,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"6\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"9\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Use of optimized algorithms:<\/strong> the use of optimized algorithms for distributed computing can reduce processing time and, consequently, the associated costs. Efficient algorithms can minimize the amount of data transferred between nodes, optimize the use of memory and reduce the consumption of computing resources. This results in faster and more efficient processing, which can reduce operating costs.<\/span> \u00a0<\/li><\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-82f6a06 elementor-widget elementor-widget-image\" data-id=\"82f6a06\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"437\" src=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-26-1-1024x559.png\" class=\"attachment-large size-large wp-image-21019\" alt=\"\" srcset=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-26-1-1024x559.png 1024w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-26-1-300x164.png 300w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-26-1-768x419.png 768w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/MicrosoftTeams-image-26-1.png 1531w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-cb42542 elementor-widget elementor-widget-text-editor\" data-id=\"cb42542\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">In addition to the computing resources associated with distributed solutions, there is another important aspect: disc storage space. There are two actions that can be taken to improve storage volume and optimize data processing:\u00a0<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><ul><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"7\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"10\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Data Compression:<\/strong> data compression is a process of reducing the size of files or data, causing it to occupy less storage space or to be transmitted more efficiently over communication networks. It reduces the number of bits needed to stand for information without losing essential data; and<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><\/ul><ul><li data-leveltext=\"\uf0b7\" data-font=\"Symbol\" data-listid=\"7\" data-list-defn-props=\"{&quot;335552541&quot;:1,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769226&quot;:&quot;Symbol&quot;,&quot;469769242&quot;:[8226],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;\uf0b7&quot;,&quot;469777815&quot;:&quot;hybridMultilevel&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>File format:<\/strong> choosing the right file format can have a significant impact on the performance and efficiency of data processing in a big data environment. Formats like Parquet, Avro and ORC are commonly used, as they are optimized for compression, efficient reading and schema support.<\/span>\u00a0<\/li><\/ul>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8f61c30 elementor-widget elementor-widget-heading\" data-id=\"8f61c30\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Democratized integrations and access <\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-8bd46cf elementor-widget elementor-widget-text-editor\" data-id=\"8bd46cf\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">With a diversity of projects and data demands arising within the corporate environment, a system that enables different teams to work simultaneously and with total control of their processes is essential for the organic growth of a data-driven company. The more acculturated your teams are, the better the results will be, which can be done through specific training, workshops on key technologies, etc.\u00a0<\/span><span data-ccp-props=\"{&quot;134233117&quot;:false,&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559685&quot;:0,&quot;335559737&quot;:0,&quot;335559738&quot;:0,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">As it is an open-source technology, Apache NiFi can be included in the software ecosystem in different ways, but its most optimized form is via Kubernetes, so it can be delivered in on-premises and cloud environments.\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">For democratic access to these environments, we can use native cloud services as part of the integrations. An example of this is the integration for authenticating the Apache NiFi environment with the <span style=\"text-decoration: underline;\"><span style=\"color: #0000ff; text-decoration: underline;\"><a style=\"color: #0000ff; text-decoration: underline;\" href=\"https:\/\/aws.amazon.com\/cognito\/?nc1=h_ls\" target=\"_blank\" rel=\"noopener\">Amazon Cognito<\/a><\/span><\/span> service.<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-f3cb0f3 elementor-widget elementor-widget-heading\" data-id=\"f3cb0f3\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h3 class=\"elementor-heading-title elementor-size-default\">Amazon Cognito x Apache NiFi <\/h3>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-daa5a5f elementor-widget elementor-widget-text-editor\" data-id=\"daa5a5f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">Amazon Cognito is an identity and access management service offered by Amazon Web Services (AWS). It is designed to help developers easily add authentication, authorization and user management features to their web and mobile applications, allowing them to focus their efforts on developing the application itself, rather than worrying about the complexity of building and maintaining authentication systems.<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-e8ffd5c elementor-widget elementor-widget-image\" data-id=\"e8ffd5c\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<img decoding=\"async\" src=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/novas_translate-03.png\" title=\"novas_translate-03\" alt=\"apache-nifi\" loading=\"lazy\" \/>\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-7fee017 elementor-widget elementor-widget-text-editor\" data-id=\"7fee017\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">The flow shown in the image has all its authorization and authentication communication based on the <span style=\"text-decoration: underline;\"><span style=\"color: #0000ff; text-decoration: underline;\"><a style=\"color: #0000ff; text-decoration: underline;\" href=\"https:\/\/auth0.com\/docs\/authenticate\/protocols\/openid-connect-protocol\" target=\"_blank\" rel=\"noopener\">OIDC protocol<\/a><\/span><\/span>. It shows how the authentication process works to release a user to the Apache NiFi interface via a private network, passing through the Route 53 DNS resolver. When accessing the NiFi interface, if the user is not authenticated, they are directed to the Amazon Cognito login screen. By logging in, the NiFi graphical interface will bring up different possibilities for system administration and use (ACL) actions.<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-bc093f9 elementor-widget elementor-widget-image\" data-id=\"bc093f9\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"image.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t\t\t\t<figure class=\"wp-caption\">\n\t\t\t\t\t\t\t\t\t\t\t<a href=\"https:\/\/nifi.apache.org\/assets\/images\/flow.png\" target=\"_blank\" rel=\"noopener\">\n\t\t\t\t\t\t\t<img loading=\"lazy\" decoding=\"async\" width=\"800\" height=\"420\" src=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/flow-1024x537.png\" class=\"attachment-large size-large wp-image-21017\" alt=\"\" srcset=\"https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/flow-1024x537.png 1024w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/flow-300x157.png 300w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/flow-768x403.png 768w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/flow-1536x806.png 1536w, https:\/\/elogroup.com\/wp-content\/uploads\/2023\/08\/flow.png 2048w\" sizes=\"(max-width: 800px) 100vw, 800px\" \/>\t\t\t\t\t\t\t\t<\/a>\n\t\t\t\t\t\t\t\t\t\t\t<figcaption class=\"widget-image-caption wp-caption-text\">Click on the image for a larger version<\/figcaption>\n\t\t\t\t\t\t\t\t\t\t<\/figure>\n\t\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-9ad5bbd elementor-widget elementor-widget-heading\" data-id=\"9ad5bbd\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Use case <\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-08092fb elementor-widget elementor-widget-text-editor\" data-id=\"08092fb\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">In one use case within an organization, we consume 967 data tables totaling 9.6 terabytes of critical information. The challenge is clear: how can this organization move, transform and manage this massive amount of data reliably and efficiently? This is where Apache NiFi comes in, an open-source tool designed to automate the flow of data between different systems.\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">Apache NiFi was deployed as an end-to-end solution to meet the complex demands of this organization. Let\u2019s see how it played a key role in meeting this challenge:<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><ol><li data-leveltext=\"%1.\" data-font=\"Arial\" data-listid=\"8\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;,&quot;469778510&quot;:&quot;default&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Simplified collection:<\/strong> Apache NiFi enabled the organization to collect data from a wide variety of sources, from traditional databases to real-time data feeds. Its intuitive visual interface made setting up these collection flows an accessible task even for less technical users.<\/span><\/li><li data-leveltext=\"%1.\" data-font=\"Arial\" data-listid=\"8\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;,&quot;469778510&quot;:&quot;default&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Dynamic transformation:<\/strong> with the ability to apply real-time transformations to data, NiFi has made it easier to clean and format data according to the organization&#8217;s needs. This reduced the load on the target systems, ensuring that only the relevant data was transmitted.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559737&quot;:-20,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"%1.\" data-font=\"Arial\" data-listid=\"8\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;,&quot;469778510&quot;:&quot;default&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Security in focus:<\/strong> considering the sensitive value of the data, security was of the utmost importance. NiFi offered advanced security features, including encryption, authentication and granular authorization, ensuring that data in transit remained protected.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559737&quot;:-20,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"%1.\" data-font=\"Arial\" data-listid=\"8\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;,&quot;469778510&quot;:&quot;default&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Robust scalability:<\/strong> as the organization grew, so did the volume of data. Apache NiFi handled this elegantly, allowing resources to be expanded horizontally to accommodate larger data loads, without compromising performance.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559737&quot;:-20,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><li data-leveltext=\"%1.\" data-font=\"Arial\" data-listid=\"8\" data-list-defn-props=\"{&quot;335552541&quot;:0,&quot;335559685&quot;:720,&quot;335559991&quot;:360,&quot;469769242&quot;:[65533,0],&quot;469777803&quot;:&quot;left&quot;,&quot;469777804&quot;:&quot;%1.&quot;,&quot;469777815&quot;:&quot;multilevel&quot;,&quot;469778510&quot;:&quot;default&quot;}\" aria-setsize=\"-1\" data-aria-posinset=\"1\" data-aria-level=\"1\"><span data-contrast=\"auto\"><strong>Real-time monitoring:<\/strong> visibility is crucial in Big Data scenarios. NiFi provided a real-time dashboard, allowing administrators to check the flow of data, identify bottlenecks and respond to problems quickly.<\/span><span data-ccp-props=\"{&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559739&quot;:160,&quot;335559740&quot;:360}\">\u00a0<\/span><\/li><\/ol><p><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559685&quot;:0,&quot;335559737&quot;:-20,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">The result was impressive: Apache NiFi enabled the organization not only to overcome the challenge of dealing with 9.6 terabytes of data distributed across 967 tables, but also to turn this challenge into an opportunity to optimize its processes, reduce costs and make more informed decisions based on the insights extracted from the data.\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559685&quot;:0,&quot;335559737&quot;:-20,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">In short, Apache NiFi showed its effectiveness in dealing with complex Big Data scenarios, enabling this organization to face the challenges of moving and managing large-scale data with confidence and success. With its flexibility, security and scalability, Apache NiFi stands out as a valuable tool for those looking to optimize their operations in a data-driven world.<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559685&quot;:0,&quot;335559737&quot;:-20,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-61ac07a elementor-widget elementor-widget-heading\" data-id=\"61ac07a\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"heading.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t<h2 class=\"elementor-heading-title elementor-size-default\">Conclusion: Apache NiFi as a facilitator for data ingestion \n\n  <\/h2>\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-6000d2f elementor-widget elementor-widget-text-editor\" data-id=\"6000d2f\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><span data-contrast=\"auto\">We can consider Apache NiFi a facilitator for data ingestion in organizations. It provides a series of features and functionalities that have the potential to make the way companies deal with collecting, processing and moving information more efficient. It helps guarantee data security, allows granular access control, eases collaboration between teams, enables auditing of activities, isolates environments, prevents accidental errors, empowers users with relevant access and adapts to the evolution of the business, contributing to efficiency and security in the flow of business data.\u00a0<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559685&quot;:0,&quot;335559737&quot;:-20,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p><p><span data-contrast=\"auto\">With the right direction, NiFi speeds up data integration, automates processes and enables real-time analyses, boosting operational efficiency, data-based decision-making and agile adaptation, strengthening overall business performance.<\/span><span data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559685&quot;:0,&quot;335559737&quot;:-20,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">\u00a0<\/span><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-c0ca227 elementor-widget-divider--view-line elementor-widget elementor-widget-divider\" data-id=\"c0ca227\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"divider.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t<div class=\"elementor-divider\">\n\t\t\t<span class=\"elementor-divider-separator\">\n\t\t\t\t\t\t<\/span>\n\t\t<\/div>\n\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t<div class=\"elementor-element elementor-element-ce46a81 elementor-widget elementor-widget-text-editor\" data-id=\"ce46a81\" data-element_type=\"widget\" data-e-type=\"widget\" data-widget_type=\"text-editor.default\">\n\t\t\t\t<div class=\"elementor-widget-container\">\n\t\t\t\t\t\t\t\t\t<p><em>FELIPE GOCHI works as D<span class=\"TextRun SCXW25979318 BCX0\" lang=\"EN-US\" xml:lang=\"EN-US\" data-contrast=\"auto\"><span class=\"NormalTextRun SCXW25979318 BCX0\">ata Engineering Specialist at EloGroup<\/span><\/span><span class=\"EOP SCXW25979318 BCX0\" data-ccp-props=\"{&quot;134233118&quot;:false,&quot;201341983&quot;:0,&quot;335551550&quot;:6,&quot;335551620&quot;:6,&quot;335559685&quot;:0,&quot;335559737&quot;:-20,&quot;335559739&quot;:0,&quot;335559740&quot;:360}\">.<\/span><\/em><\/p><p><em>FABIO CATEIN works as Senior Data Engineering Specialist at EloGroup.<\/em><\/p><p><em>RODRIGO BASSANI works as CTO at EloGroup.<\/em><\/p>\t\t\t\t\t\t\t\t<\/div>\n\t\t\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/div>\n\t\t\t\t\t<\/div>\n\t\t<\/section>\n\t\t\t\t<\/div>\n\t\t","protected":false},"excerpt":{"rendered":"<p>Apache NiFi is an open source data processing and orchestration tool designed to facilitate the movement of information between different systems in real time. Because it is highly scalable and fault-tolerant, it is suitable for dealing with large volumes of data in distributed environments.<\/p>\n","protected":false},"author":9,"featured_media":21011,"parent":0,"template":"","editorias":[125],"industrias-category":[147,129,133,148,134,143,149],"praticas-category":[153],"insights-category":[160],"class_list":["post-23480","insights","type-insights","status-publish","has-post-thumbnail","hentry","editorias-strategically-digital-en","industrias-category-financial-services","industrias-category-food-and-beverages","industrias-category-infraestrutura-en","industrias-category-logistics-and-transportation","industrias-category-manufatura-en","industrias-category-media-and-telecom","industrias-category-retail","praticas-category-data-engineering-big-data-en","insights-category-strategically-digital-en"],"_links":{"self":[{"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/insights\/23480","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/insights"}],"about":[{"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/types\/insights"}],"author":[{"embeddable":true,"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/users\/9"}],"version-history":[{"count":27,"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/insights\/23480\/revisions"}],"predecessor-version":[{"id":23522,"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/insights\/23480\/revisions\/23522"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/media\/21011"}],"wp:attachment":[{"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/media?parent=23480"}],"wp:term":[{"taxonomy":"editorias","embeddable":true,"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/editorias?post=23480"},{"taxonomy":"industrias-category","embeddable":true,"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/industrias-category?post=23480"},{"taxonomy":"praticas-category","embeddable":true,"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/praticas-category?post=23480"},{"taxonomy":"insights-category","embeddable":true,"href":"https:\/\/elogroup.com\/en\/wp-json\/wp\/v2\/insights-category?post=23480"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}