{"id":31255,"date":"2020-10-05T14:03:45","date_gmt":"2020-10-05T12:03:45","guid":{"rendered":"https:\/\/fotc.com\/?p=31255"},"modified":"2023-12-21T15:04:11","modified_gmt":"2023-12-21T14:04:11","slug":"dataflow-flex-templates","status":"publish","type":"post","link":"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/","title":{"rendered":"Elastyczne tworzenie szablon\u00f3w dzi\u0119ki Dataflow Flex Templates w Google Cloud"},"content":{"rendered":"\n<div class=\"wp-block-yoast-seo-table-of-contents yoast-table-of-contents collapsed\"><h2> Spis tre\u015bci <\/h2><ul><li><a href=\"#h-udost-pnianie-pipeline-w-dataflow-z-klasycznymi-szablonami\" data-level=\"2\">Udost\u0119pnianie pipeline&#8217;\u00f3w Dataflow z klasycznymi szablonami<\/a><\/li><li><a href=\"#h-dopasowanie-pipeline-do-potrzeb-analizy-dzi-ki-dataflow-flex-templates\" data-level=\"2\">Dopasowanie pipeline do potrzeb analizy dzi\u0119ki Dataflow Flex Templates<\/a><\/li><\/ul><\/div>\n\n\n\n<p><span>Analiza danych stanowi wa\u017cny element rozwoju wielu organizacji. To obszar, w kt\u00f3rym znalezienie wsp\u00f3lnego j\u0119zyka mi\u0119dzy technologi\u0105 a biznesem jest niezwykle wa\u017cne do efektywnego dzia\u0142ania.<\/span><\/p>\n\n\n\n<p><span>W wielu przypadkach dzia\u0142y analiz biznesowych korzystaj\u0105 z przygotowanych przez developer\u00f3w pipeline\u2019\u00f3w. Pipeline to cykl szeregowego przetwarzania danych, gdzie dane wyj\u015bciowe jednego elementu s\u0105 danymi wej\u015bciowymi elementu nast\u0119pnego.&nbsp;<\/span><\/p>\n\n\n\n<!--more-->\n\n\n\n<p><span>Do niedawna wykorzystanie przez biznes pracy developer\u00f3w by\u0142o do\u015b\u0107 k\u0142opotliwe \u2013 <strong>uruchomienie pipeline\u2019u wymaga\u0142o instalacji i konfiguracji \u015brodowiska programistycznego wraz ze wszystkimi zale\u017cno\u015bciami (np. bibliotekami) po stronie u\u017cytkownika oraz wprowadzenia zmian w kodzie \u017ar\u00f3d\u0142owym<\/strong>. A nie ka\u017cdy analityk biznesowy ma wiedz\u0119 programistyczn\u0105, by sprawnie poradzi\u0107 sobie z tym zadaniem.<\/span><\/p>\n\n\n\n<p><span>Grafika prezentuje proces uruchamiania pipeline przed wprowadzeniem przez <a href=\"https:\/\/fotc.com\/pl\/google-cloud-platform\/\" target=\"_blank\" aria-label=\" (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"ek-link\">Google Cloud<\/a> rozwi\u0105zania, jakim s\u0105 szablony <strong><a href=\"https:\/\/fotc.com\/pl\/blog\/dataflow-elastyczne-przeplywy-danych\/\" target=\"_blank\" aria-label=\" (opens in a new tab)\" rel=\"noreferrer noopener\" class=\"ek-link\">Dataflow<\/a>:<\/strong><\/span><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/fotc.com\/app\/uploads\/2021\/08\/dataflow-before-templates-1024x293.png\" alt=\"dataflow flex templates\" class=\"wp-image-61417\"\/><\/figure>\n\n\n\n<p><strong>Proces uruchamiania pipeline znacznie upro\u015bci\u0142y szablony \u2013 classic templates i od niedawna Flex Templates.<\/strong><\/p>\n\n\n\n<h2 id=\"h-udost-pnianie-pipeline-w-dataflow-z-klasycznymi-szablonami\"><span>Udost\u0119pnianie pipeline&#8217;\u00f3w Dataflow z klasycznymi szablonami<\/span><\/h2>\n\n\n\n<p><span>Dotychczas Google dawa\u0142o mo\u017cliwo\u015b\u0107 wykorzystania klasycznych szablon\u00f3w (classic templates). Funkcjonalno\u015b\u0107 pozwala developerowi uruchomi\u0107 pipeline w \u015brodowisku programistycznym, gdzie nast\u0119pnie konwertuje go do szablonu i zamieszcza w Cloud Storage. <strong>U\u017cytkownik mo\u017ce uruchomi\u0107 szablon bez konieczno\u015bci instalowania i konfigurowania \u015brodowiska programistycznego.<\/strong><\/span><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/fotc.com\/app\/uploads\/2021\/08\/dataflow-classic-templates.png\" alt=\"dataflow flex templates\" class=\"wp-image-61424\"\/><\/figure>\n\n\n\n<p><span><strong>Klasyczne szablony posiadaj\u0105 jednak kilka ogranicze\u0144<\/strong> \u2013 m.in. nie daj\u0105 u\u017cytkownikowi mo\u017cliwo\u015bci wprowadzania zmian czy samodzielnego dopasowania do potrzeb analizy (np. w kwestii wyboru \u017ar\u00f3d\u0142a danych wyj\u015bciowych), co dalej wymaga\u0142o od developer\u00f3w tworzenia r\u00f3\u017cnych wariant\u00f3w pipeline\u2019\u00f3w; klasyczne szablony s\u0105 te\u017c uzale\u017cnione od interfejsu ValueProvider Apache Beam, kt\u00f3re posiada ograniczon\u0105 liczb\u0119 \u017ar\u00f3de\u0142 wej\u015bcia i wyj\u015bcia. Analityka, nawet z u\u0142atwieniem w postaci <\/span><i><span>classic templates<\/span><\/i><span>, wci\u0105\u017c by\u0142a ci\u0119\u017ckim kawa\u0142kiem chleba, anga\u017cuj\u0105cym developer\u00f3w i znacznie ograniczaj\u0105cym analityk\u00f3w.<\/span><\/p>\n\n\n\n<h2 id=\"h-dopasowanie-pipeline-do-potrzeb-analizy-dzi-ki-dataflow-flex-templates\"><span>Dopasowanie pipeline do potrzeb analizy dzi\u0119ki Dataflow Flex Templates<\/span><\/h2>\n\n\n\n<p><span>Z pocz\u0105tkiem pa\u017adziernika 2020 Google rozszerzy\u0142o mo\u017cliwo\u015bci udost\u0119pniania i wykorzystania pipeline\u2019\u00f3w dzi\u0119ki Flex Templates.<\/span><\/p>\n\n\n\n<p><strong>Flex Templates zapewnia wi\u0119ksz\u0105 elastyczno\u015b\u0107 w por\u00f3wnaniu z szablonami klasycznymi.<\/strong> Podobnie jak classic templates, Flex Templates r\u00f3wnie\u017c daje mo\u017cliwo\u015b\u0107 uruchomienia pipeline\u2019u bez konieczno\u015bci instalowania \u015brodowiska programistycznego. Ponadto pozwala analitykom wprowadza\u0107 zmiany w szablonach \u2013 np. wybiera\u0107 dowolne \u017ar\u00f3d\u0142o danych wej\u015bciowych i wyj\u015bciowych (<i>source\/sink<\/i>), co zdejmuje z developer\u00f3w konieczno\u015b\u0107 tworzenia kilku wariant\u00f3w szablon\u00f3w. Flex Templates usuwa te\u017c zale\u017cno\u015b\u0107 od interfejsu ValueProvider, co pozwala korzysta\u0107 z wi\u0119kszej liczby \u017ar\u00f3de\u0142.<\/p>\n\n\n\n<p><span>Schemat tworzenia i udost\u0119pniania Flex Templates wygl\u0105da nast\u0119puj\u0105co:<\/span><\/p>\n\n\n\n<figure class=\"wp-block-image\"><img src=\"https:\/\/fotc.com\/app\/uploads\/2021\/08\/dataflow-flex-templates-1024x290.png\" alt=\"dataflow flex templates\" class=\"wp-image-61410\"\/><\/figure>\n\n\n\n<p><span>Developer tworzy pipeline. Zamiast zamieszcza\u0107 go w Cloud Storage, konwertuje kod lub pliki binarne do obrazu dockera i zapisuje go w Google Container Registry. Tworzy te\u017c plik ze specyfikacj\u0105 szablonu, kt\u00f3ry jest przechowywany w Cloud Storage. Konstrukcja Flex Templates umo\u017cliwia dynamiczne tworzenie diagram\u00f3w na podstawie danych wprowadzonych przez u\u017cytkownika.&nbsp;<\/span><\/p>\n\n\n\n<p><span>U\u017cytkownik mo\u017ce wprowadzi\u0107 dane i parametry oraz uruchomi\u0107 szablon z poziomu wiersza polece\u0144 gcloud, u\u017cywaj\u0105c API lub interfejsu Dataflow w konsoli Google Cloud, wskazuj\u0105c odpowiedni plik w Cloud Storage.<\/span><\/p>\n\n\n\n<p><strong>Wi\u0119cej informacji o Dataflow Flex Templates znajdziesz w filmie ni\u017cej:<\/strong><\/p>\n\n\n\n<p><iframe src=\"https:\/\/www.youtube.com\/embed\/xaoruVUt0jk\" width=\"560\" height=\"315\" frameborder=\"0\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/p>\n\n\n\n<p><span><strong>\u0179r\u00f3d\u0142o<\/strong>: <\/span><a href=\"https:\/\/cloud.google.com\/blog\/products\/data-analytics\/create-templates-from-any-dataflow-pipeline\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" class=\"ek-link\"><span>Google \u2013 Turn any Dataflow pipeline into a reusable template<\/span><\/a><\/p>\n\n\n\n<p><strong>Zobacz te\u017c:<\/strong><\/p>\n\n\n\n<ul><li><a href=\"https:\/\/fotc.com\/pl\/blog\/bigquery-co-to-jest-jak-zaczac-tutorial\/\" class=\"ek-link\">BigQuery <a href=\"https:\/\/cloud.google.com\/blog\/products\/data-analytics\/create-templates-from-any-dataflow-pipeline\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" class=\"ek-link\"><span>\u2013<\/span><\/a> co to jest i jak zacz\u0105\u0107?<\/a> [Tutorial]<\/li><li><a href=\"https:\/\/fotc.com\/pl\/blog\/kubernetes-tutorial\/\" class=\"ek-link\">Kubernetes <a href=\"https:\/\/cloud.google.com\/blog\/products\/data-analytics\/create-templates-from-any-dataflow-pipeline\" target=\"_blank\" rel=\"nofollow noopener noreferrer\" class=\"ek-link\"><span>\u2013<\/span><\/a> co to jest i jak zacz\u0105\u0107?<\/a> [Tutorial na GCP]<\/li><li><a href=\"https:\/\/fotc.com\/pl\/blog\/kubernetes-docker-swarm\/\" class=\"ek-link\">Kubernetes vs. Docker Swarm<\/a> <span>\u2013<\/span> por\u00f3wnanie platform do orkiestracji<\/li><\/ul>\n","protected":false},"excerpt":{"rendered":"<p>Analiza danych stanowi wa\u017cny element rozwoju wielu organizacji. To obszar, w kt\u00f3rym znalezienie wsp\u00f3lnego j\u0119zyka mi\u0119dzy technologi\u0105 a biznesem jest niezwykle wa\u017cne do efektywnego dzia\u0142ania. W wielu przypadkach dzia\u0142y analiz biznesowych korzystaj\u0105 z przygotowanych przez developer\u00f3w pipeline\u2019\u00f3w. Pipeline to cykl szeregowego przetwarzania danych, gdzie dane wyj\u015bciowe jednego elementu s\u0105 danymi wej\u015bciowymi elementu nast\u0119pnego.&nbsp;<\/p>\n","protected":false},"author":13,"featured_media":5525,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_editorskit_title_hidden":false,"_editorskit_reading_time":2,"_editorskit_typography_data":[],"_editorskit_blocks_typography":"","_editorskit_is_block_options_detached":false,"_editorskit_block_options_position":"{}"},"categories":[560],"tags":[107],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v16.7 (Yoast SEO v19.6) - https:\/\/yoast.com\/wordpress\/plugins\/seo\/ -->\n<title>Dataflow Flex Templates w Google Cloud | FOTC<\/title>\n<meta name=\"description\" content=\"Flex Templates pozwala tworzy\u0107 i udost\u0119pnia\u0107 pipeline&#039;y, kt\u00f3re mog\u0105 by\u0107 nast\u0119pnie dostosowane do potrzeb analizy biznesowej.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/\" \/>\n<meta property=\"og:locale\" content=\"pl_PL\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Elastyczne tworzenie szablon\u00f3w dzi\u0119ki Dataflow Flex Templates w Google Cloud\" \/>\n<meta property=\"og:description\" content=\"Flex Templates pozwala tworzy\u0107 i udost\u0119pnia\u0107 pipeline&#039;y, kt\u00f3re mog\u0105 by\u0107 nast\u0119pnie dostosowane do potrzeb analizy biznesowej.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/\" \/>\n<meta property=\"og:site_name\" content=\"FOTC\" \/>\n<meta property=\"article:published_time\" content=\"2020-10-05T12:03:45+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2023-12-21T14:04:11+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/fotc.com\/app\/uploads\/2021\/07\/GCP-Flex-Templates.png\" \/>\n\t<meta property=\"og:image:width\" content=\"1200\" \/>\n\t<meta property=\"og:image:height\" content=\"628\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Ida O\u017carowska\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/\",\"url\":\"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/\",\"name\":\"Dataflow Flex Templates w Google Cloud | FOTC\",\"isPartOf\":{\"@id\":\"https:\/\/fotc.com\/#website\"},\"datePublished\":\"2020-10-05T12:03:45+00:00\",\"dateModified\":\"2023-12-21T14:04:11+00:00\",\"author\":{\"@id\":\"https:\/\/fotc.com\/#\/schema\/person\/84a7428ea802a35722a1839225ab0dfb\"},\"description\":\"Flex Templates pozwala tworzy\u0107 i udost\u0119pnia\u0107 pipeline'y, kt\u00f3re mog\u0105 by\u0107 nast\u0119pnie dostosowane do potrzeb analizy biznesowej.\",\"breadcrumb\":{\"@id\":\"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/#breadcrumb\"},\"inLanguage\":\"pl-PL\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/\"]}],\"accessibilityFeature\":[\"tableOfContents\"]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"PL\",\"item\":\"https:\/\/fotc.com\/pl\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Elastyczne tworzenie szablon\u00f3w dzi\u0119ki Dataflow Flex Templates w Google Cloud\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/fotc.com\/#website\",\"url\":\"https:\/\/fotc.com\/\",\"name\":\"FOTC\",\"description\":\"Partner Google Cloud\",\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/fotc.com\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"pl-PL\"},{\"@type\":\"Person\",\"@id\":\"https:\/\/fotc.com\/#\/schema\/person\/84a7428ea802a35722a1839225ab0dfb\",\"name\":\"Ida O\u017carowska\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"pl-PL\",\"@id\":\"https:\/\/fotc.com\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/e05ba028ea67bd9a1d51e046729088a9?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/e05ba028ea67bd9a1d51e046729088a9?s=96&d=mm&r=g\",\"caption\":\"Ida O\u017carowska\"},\"description\":\"Ida has a Master's Degree in Communication Management from the University of Wroc\u0142aw. For over 7 years, she has been creating content that connects the worlds of technology and business. At FOTC, she is a Content Manager.\",\"url\":\"https:\/\/fotc.com\/pl\/blog\/author\/idaozarowska\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Dataflow Flex Templates w Google Cloud | FOTC","description":"Flex Templates pozwala tworzy\u0107 i udost\u0119pnia\u0107 pipeline'y, kt\u00f3re mog\u0105 by\u0107 nast\u0119pnie dostosowane do potrzeb analizy biznesowej.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/","og_locale":"pl_PL","og_type":"article","og_title":"Elastyczne tworzenie szablon\u00f3w dzi\u0119ki Dataflow Flex Templates w Google Cloud","og_description":"Flex Templates pozwala tworzy\u0107 i udost\u0119pnia\u0107 pipeline'y, kt\u00f3re mog\u0105 by\u0107 nast\u0119pnie dostosowane do potrzeb analizy biznesowej.","og_url":"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/","og_site_name":"FOTC","article_published_time":"2020-10-05T12:03:45+00:00","article_modified_time":"2023-12-21T14:04:11+00:00","og_image":[{"width":1200,"height":628,"url":"https:\/\/fotc.com\/app\/uploads\/2021\/07\/GCP-Flex-Templates.png","type":"image\/png"}],"author":"Ida O\u017carowska","twitter_card":"summary_large_image","schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/","url":"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/","name":"Dataflow Flex Templates w Google Cloud | FOTC","isPartOf":{"@id":"https:\/\/fotc.com\/#website"},"datePublished":"2020-10-05T12:03:45+00:00","dateModified":"2023-12-21T14:04:11+00:00","author":{"@id":"https:\/\/fotc.com\/#\/schema\/person\/84a7428ea802a35722a1839225ab0dfb"},"description":"Flex Templates pozwala tworzy\u0107 i udost\u0119pnia\u0107 pipeline'y, kt\u00f3re mog\u0105 by\u0107 nast\u0119pnie dostosowane do potrzeb analizy biznesowej.","breadcrumb":{"@id":"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/#breadcrumb"},"inLanguage":"pl-PL","potentialAction":[{"@type":"ReadAction","target":["https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/"]}],"accessibilityFeature":["tableOfContents"]},{"@type":"BreadcrumbList","@id":"https:\/\/fotc.com\/pl\/blog\/dataflow-flex-templates\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"PL","item":"https:\/\/fotc.com\/pl\/"},{"@type":"ListItem","position":2,"name":"Elastyczne tworzenie szablon\u00f3w dzi\u0119ki Dataflow Flex Templates w Google Cloud"}]},{"@type":"WebSite","@id":"https:\/\/fotc.com\/#website","url":"https:\/\/fotc.com\/","name":"FOTC","description":"Partner Google Cloud","potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/fotc.com\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"pl-PL"},{"@type":"Person","@id":"https:\/\/fotc.com\/#\/schema\/person\/84a7428ea802a35722a1839225ab0dfb","name":"Ida O\u017carowska","image":{"@type":"ImageObject","inLanguage":"pl-PL","@id":"https:\/\/fotc.com\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/e05ba028ea67bd9a1d51e046729088a9?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/e05ba028ea67bd9a1d51e046729088a9?s=96&d=mm&r=g","caption":"Ida O\u017carowska"},"description":"Ida has a Master's Degree in Communication Management from the University of Wroc\u0142aw. For over 7 years, she has been creating content that connects the worlds of technology and business. At FOTC, she is a Content Manager.","url":"https:\/\/fotc.com\/pl\/blog\/author\/idaozarowska\/"}]}},"_links":{"self":[{"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/posts\/31255"}],"collection":[{"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/users\/13"}],"replies":[{"embeddable":true,"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/comments?post=31255"}],"version-history":[{"count":0,"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/posts\/31255\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/media\/5525"}],"wp:attachment":[{"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/media?parent=31255"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/categories?post=31255"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/fotc.com\/pl\/wp-json\/wp\/v2\/tags?post=31255"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}