@sahilpaudel wrote:

def create(conn, params) do
    conn 
      |> render_json_response(bulk_convertor(params))
  end

  def bulk_convertor(%{"file" => %Plug.Upload{content_type: "text/csv", filename: basename, path: fullpath}}) do
    list = [~w(old, new)]
    fullpath
    |> read_csv(list)
    |> form_response
  end

  def read_csv(path, list) do
    if path |> File.stream! |> Enum.count <= 50 do
      path
        |> File.stream!
        |> Parser.parse_stream
        |> Stream.map(fn [arg] -> %{url: arg} end)
        |> iterate_urls(list)
    else
      %{error: %{message: "Limit Exceeded"}}
    end
  end

  def iterate_urls(mapped_stream, list) do
    mapped_stream
      |> Enum.each(fn(url) -> url_convertor(url, list) end)
    %{response: %{message: "Request Initiated"}}
  end

  def url_convertor(%{url: val}, list) do
    with {:ok, %Route{} = route} <- App.create_route(%{"url" => val}) do
      route
        |> App.redirect_url
        |> add_to_list(route, list)
    end
  end

  def add_to_list(new_url, %{url: original_url}, list) do
    list = List.insert_at(list, -1, ~w(#{original_url}, #{new_url}))
    list
  end

The list content is not updated whenever I try to print it somewhere using IO.inspect it shows the initial list. I am trying to update the content of the list based on the data parsed from the csv which i am parsing using the nimble_csv library of elixir.

If I try to print the list inside

add_to_list

only two rows are visible the header that i added initially and last row from the CSV.

Is there anything I am doing wrong here.

Posts: 8

Participants: 5

Read full topic

@ashneyderman wrote:

Hi, there!

I am having a hard time using fragment to query MySQL JSON field

from([channel] in query, where: fragment("metadata->>'$.?' = ?", ^key, ^value))

metadata is my JSON field. When I run this code I get MySQL syntax error.

If I hard-code JSON key

from([channel] in query, where: fragment("metadata->>'$.test0' = ?", ^value))

It works but since this is free form JSON I have no prior knowledge of what keys there are inside JSON.

If I do this

from([channel] in query, where: fragment("metadata->>'$.#{key}' = ?", ^value))

I get compilation errors.

Any ideas on what might be a proper solution to this?

Posts: 3

Participants: 2

Read full topic

@thiagomajesk wrote:

Cheers everyone!

I’m finding incredibly hard to find a Whirlpool2 hashing implementation. Could someone point me in the right direction?

I couldn’t find an Elixir library that supports it so far and the closest I got to finding something useful was the C, Go and Ruby (C wrapper) versions.

PS.: I received this specification from the owner of a platform I have to integrate with and I need to generate password hashes in the same way his service does. I couldn’t find references for “Whirlpool2” exactly, so I’m guessing is the second revision of the algorithm (Whirlpool-T).

Posts: 1

Participants: 1

Read full topic

@vlad1010 wrote:

Hi everyone, sorry for the dumb question. But I am a newbie in elixir came from golang.

I really love elixir and FP style, I try to implement some basic feature and had an issue with race conditions on channel connection. For example, in golang I resolved it with Mutex, but it’s not FP style.

Initial problem:
Several channel connections, waiting for the group of random amount for example 5. And then randomly shuffle people. In this case, I had some problems with “race condition”. Same people to different groups etc.

What is the right idiomatic way to solve this in Elixir?

Thank you for your help. And sorry again for a dumb question.

Posts: 2

Participants: 2

Read full topic

@sahilpaudel wrote:

Below is the code snippet

defmodule App.JsonResponseView do
  defmacro __using__(_optional) do
    quote do
      use AppWeb, :controller

      def render_json_response(conn, {:error, errors}) do
        conn
        |> put_status(:unprocessable_entity)
        |> json(errors)
      end
    
      def render_json_response(conn, {:ok, result}) do
        conn 
        |> json(result)
      end
    end
  end
end

Which I am trying to use in

defmodule AppWeb.BulkController do
  use App.JsonResponseView

  alias App.CSVParser
  # write code to check if file was sent

  def create(conn, params) do
    conn 
      |> render_json_response(CSVParser.call(params))
  end
end

I want to use

render_json_response(CSVParser.call(params))

but it gives me undefined or private error.

Posts: 2

Participants: 2

Read full topic

@mat-hek wrote:

In our library we have two kinds of debug logs:

1. the ones that are printed very often and thus they usually introduce significant overhead and are hard to reason about, but sometimes they’re useful
2. the ones that are not printed that often but still can pollute the output in some cases

I’d like to distinguish them and be able to configure whether they should be printed or not. If possible, the silenced ones should be purged at the compile.

I know the logger can be configured to purge logs from particular modules and functions, and while it’s a nice possibility that I’d like to leave available, I think it’s not a good idea to require the lib users to explicitly list all the functions from either category.

Posts: 8

Participants: 3

Read full topic

@sahilpaudel wrote:

I am trying to send attachment inside email from my phoenix application using Bamboo.

Library used:
nimble_csv
bamboo
bamboo_smtp

The attachment is in-memory data that I am getting after parsing the CSV and making some changes into it. I don’t want to save the CSV file so using the in-memory approach.

def dump_to_csv(list) do
    [~w(ORIGINAL_URL NEW_URL)] ++ list
      |> ShortnerParser.dump_to_iodata()
      |> prepare_mail
      |> Mailer.deliver_later
  end

def prepare_mail(data) do
    attachment = %Bamboo.Attachment{content_type: "application/octet-stream", filename: "shortner.csv", data: data}
    Email.send_csv("myemail@gmail.com", attachment)
  end

Error I am getting:

[error] Task #PID<0.1556.0> started from #PID<0.1492.0> terminating
** (FunctionClauseError) no function clause matching in Base.encode64/2
    (elixir) lib/base.ex:371: Base.encode64([["ORIGINAL_URL", 44, "NEW_URL", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/ldhkRf", 10], ["https://google.com", 44, "https://localhost:4000/n594Hy", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/Kl67G5", 10], ["https://google.com", 44, "https://localhost:4000/Ct17TM", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/w1gX_x", 10], ["https://google.com", 44, "https://localhost:4000/1lqoLr", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/R_u3Yq", 10], ["https://google.com", 44, "https://localhost:4000/elXe4V", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/13hn32", 10], ["https://google.com", 44, "https://localhost:4000/j9uiKL", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/c3pzUB", 10], ["https://google.com", 44, "https://localhost:4000/Yhb7Kv", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/19PS7z", 10], ["https://google.com", 44, "https://localhost:4000/LvHqO1", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/teJagz", 10], ["https://google.com", 44, "https://localhost:4000/MBYX_e", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/DfpQ25", 10], ["https://google.com", 44, "https://localhost:4000/jLFq6P", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/xM7Jg3", 10], ["https://google.com", 44, "https://localhost:4000/IM2RG6", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/1X2IZ8", 10], ["https://google.com", 44, "https://localhost:4000/KKo0kt", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/YvTW_p", 10], ["https://google.com", 44, "https://localhost:4000/iwLt7L", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/DoIvwE", 10], ["https://google.com", 44, "https://localhost:4000/ZMe3dE", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/CGEIa1", 10], ["https://google.com", 44, "https://localhost:4000/Z6fPMT", 10], ["https:pharmeasy.in", 44, "invalid_url", 10], ["https://facebook.com", 44, "https://localhost:4000/zYpNgG", 10], ["https://google.com", 44, "https://localhost:4000/MYQdRu", 10], ["https:pharmeasy.in", 44, "invalid_url", ...], ["https://facebook.com", 44, ...], ["https://google.com", ...], [...], ...], [])
    (bamboo_smtp) lib/bamboo/adapters/smtp_adapter.ex:214: Bamboo.SMTPAdapter.add_attachment_body/2
    (elixir) lib/enum.ex:1314: Enum."-map/2-lists^map/1-0-"/2
    (bamboo_smtp) lib/bamboo/adapters/smtp_adapter.ex:233: Bamboo.SMTPAdapter.add_attachments/3
    (bamboo_smtp) lib/bamboo/adapters/smtp_adapter.ex:278: Bamboo.SMTPAdapter.body/1
    (bamboo_smtp) lib/bamboo/adapters/smtp_adapter.ex:360: Bamboo.SMTPAdapter.to_gen_smtp_message/1
    (bamboo_smtp) lib/bamboo/adapters/smtp_adapter.ex:76: Bamboo.SMTPAdapter.deliver/2
    (elixir) lib/task/supervised.ex:89: Task.Supervised.do_apply/2
    (stdlib) proc_lib.erl:249: :proc_lib.init_p_do_apply/3
Function: #Function<0.73693256/0 in Bamboo.TaskSupervisorStrategy.deliver_later/3>
    Args: []

Posts: 3

Participants: 2

Read full topic

@brettbeatty wrote:

I have an app getting a lot of timeouts while resolving GraphQL queries, and I’m trying to identify the culprit query or queries. My logs don’t have much context right now, so I’m looking to add the current object and field identifiers to the logger metadata.

My best idea so far is to have a middleware put/delete the appropriate metadata. Here’s what I have so far:

defmodule MyAppWeb.GraphQL.Schema do
  use Absinthe.Schema
  alias MyAppWeb.GraphQL.Middleware.LoggerMetadata

  ...

  def middleware(middleware, field, object) do
    Enum.flat_map(middleware, fn
      current_middleware = {{Absinthe.Resolution, :call}, _resolver} ->
        [
          {{LoggerMetadata, :put}, {object.identifier, field.identifier}},
          current_middleware,
          {{LoggerMetadata, :delete}, nil}
        ]

      current_middleware ->
        current_middleware
    end)
  end
end

defmodule MyAppWeb.GraphQL.Middleware.LoggerMetadata do
  @spec put(Absinthe.Resolution.t(), {atom(), atom()}) :: Absinthe.Resolution.t()
  def put(resolution, {object_id, field_id}) do
    Logger.metadata(graphql_object: object_id, graphql_field: field_id)

    resolution
  end

  @spec delete(Absinthe.Resolution.t(), none()) :: Absinthe.Resolution.t()
  def delete(resolution, _) do
    Logger.metadata(graphql_object: nil, graphql_field: nil)

    resolution
  end
end

This works, but the Enum.flat_map/2 solution seems brittle to me: if the behavior of the resolve macro ever changed, this would no longer work.

Is there a better way about this? I’d love to just be able to change the behavior of the resolve macro to return all three middlewares, but I’m assuming that’s not possible (the source code has been hard to follow). I don’t want to create a custom resolve macro; it seems like too much of a chore to enforce its use over the regular resolve macro.

Posts: 2

Participants: 2

Read full topic

@fireproofsocks wrote:

This is an open question, but it’s something I’ve been thinking about as I play around with processes and state for a game. (Background: I’ve got multiple “player” modules/processes competing in a “game” of survival).

Sometimes, I want certain bits of data to be “private”, so that other processes cannot read it or write to it. In my case, a player module might need to know some things about another player module in order to “compete” against it, but as soon as you know the other player’s PID, it might be possible to see data that you’re not supposed to see or worse, to overwrite data. (This scenario might make more sense if I asked 100 people to each contribute their own player module to a “tournament” – the code won’t work if they don’t implement the defined behaviours/callbacks, but there’s no guarantee that people won’t peek into the PIDs).

So my question is: What strategies are there in Elixir/Erlang for keeping data private? The only thing that comes to mind is the simple (albeit sometimes tedious) structuring of code that would ensure:

1. only passing along bits of data that is safe to be read
2. being very careful when dispatching execution that updates data
3. guarding PIDs carefully (i.e. don’t reveal them unnecessarily)

Related question: Is there any way to enumerate PIDs? Just eyeballing my simple spawned processes, it looks the the PIDs are very much incremental. How would one go about about enumerating PIDs? Coming from a cybersecurity background, it doesn’t appear that it would be difficult to start guessing PIDs and eventually find one that I could use to read or update data (e.g. on another player, in the case of my game). I could rework the “game” code in my case so that player processes were stopped and started to minimize peeking, but it wouldn’t eliminate the possibility (and it would vastly complicate/slow-down the code).

Thoughts?

Posts: 3

Participants: 3

Read full topic

@Softknobs wrote:

Hi,

I can’t tell if this is related to Elixir itself or the implementation of the Phoenix.LiveComponent module (from LiveView) I am using. Also, I am not familiar with Behaviours yet and new to Elixir.

In my code, I tried to refactor this code (this is a simplified example):

defmodule My.Module do
  use Phoenix.LiveComponent

# update is a behaviour (@callback) from  LiveComponent, 
# "assign" function adds values to the socket.assigns map
def update(assigns, socket) do
  socket |> assign(:value, assigns.value)
end

to this:

defmodule My.Module do
  use Phoenix.LiveComponent

# "assign" function adds values to the socket.assigns map
defp default_values(assigns, socket) do
  socket |> assign(:value, assigns.value)
end

 # update is a behaviour from  LiveComponent
def update(assigns, socket) do
  socket |> default_values(assigns)
end

The former code works, while the latter fails with an error (not Elixir) complaining that :value is not found in socket. But looking at the code, it is not true because default_values function adds :value to the socket just like the former code, socket |> assign(:value, assigns.value), did it.

Then I made default_values public instead of private, and the refactored code started working like the former version. I am a bit puzzled because if is a accessibility problem, I would have expected either:

• a compiler error telling the behaviour cannot access a private function inside the module that uses it
• a runtime error telling the function default_values is undefined or private.

I did not expect the code would proceed “normally” ignoring the private function (or silently catching the problem, or whatever happens here…) leading to a crash later because, in the end, the function was not applied to the data.
Actually it worries me because in this particular case the program crashes later because some requirements are not met. This not always the case, and it would have been very hard to figure out one function was not applied without crashing in other situations.

Does anybody have an idea of what is happening in this case? Does it have to do with Elixir itself or could it be the way the “used” module is implemented?

Posts: 6

Participants: 3

Read full topic

@domvas wrote:

Hello,
I’ve got a question regarding documenting functions that lies inside a __using__ block.
These functions are not listed in documentation, which is understandable because they don’t really exists yet.
But for my particular case it would be very interesting to have doc about them for end users.
Currently my module is something like this:

defmodule UsingModule do
  defmacro __using__(opts) do
    quote bind_quoted: [opts: opts] do
      def do_stuff(arg) do 
        OtherModule.do_stuff(__MODULE__, arg)
      end
      
      def do_more_stuff(arg1, arg2) do 
        OtherModule.do_more_stuff(__MODULE__, arg1, arg2)
      end
    end
end

and my goal is to document:

• UsingModule.do_stuff/1
• UsingModule.do_more_stuff/2

For now, the only working solution I’ve found is… to add callback for each of these functions, like this:

@doc """
  Mighty doc for do_stuff
  """
  @callback do_stuff(String.t) :: :ok | {:error, String.t}
  
  @doc """
  Great doc for do_more_stuff
  """
  @callback do_more_stuff(String.t, integer) :: :ok | {:error, String.t}

This is a bit strange as I don’t want a behaviour…
Is this the way to go or is there a better way to achieve what I want?

thank you

Posts: 4

Participants: 4

Read full topic

@seb3s wrote:

Hello,

What is the best way to enforce security when using liveview. Right now, a user is able to modify the DOM phx-xxx attributes by simply using developer tools and as a consequence may trigger some unwanted (and potentially very dangerous) effects server side.

It seems to me right now that I must recheck (post tag generation) in every handle_event calls that the action is really permitted for the socket asking it.

Is there something I missed that I can use ? I really would like this to be enforced by the system so to be more reliable.
TIA,
Sébastien

Posts: 3

Participants: 2

Read full topic

@zamith wrote:

Hi,

I’m trying to convert the following query to Ecto:

select * from unnest(ARRAY['one', 'two', 'not-there']) except select id from table;

I haven’t been able to get the unnest fragment to compile. Any help?

Posts: 6

Participants: 3

Read full topic

@Adzz wrote:

I have an app that has a domain layer in between the database and graphql.

It goes Graphql resovler -> hits db or wherever to get data -> passes that to the domain which is just pure functions.

The result of the domain functions is what is returned from the resolvers. This is okay, but we get into problems when trying to use dataloader. Imagine:

object :author do
  field(:name, :string)
end

object :post do
  field(:name, :string)
  field(:author, :author) do
    resolve(fn parent, _, _ ->
      
    end
  end
end

query do
  field(:posts, list_of(:posts)) do
    resolve(fn %{id: id}, _ -> 
       Db.get_post(Post, id)
       # Because this returns an embedded schema, the Ecto dataloader source wont work.
       |> Domain.new_post()
    end
  end
end

Is there a way to get the requests to the db batched with dataloader, but still return domain structs from the resolvers?

Posts: 1

Participants: 1

Read full topic

@david234 wrote:

I have this form

                    <%= form_for @conn, Routes.admin_products_path(@conn, :create_product), [as: :create_product, id: "product-form", class: "mt-5 mb-5 login-input", method: :post, multipart: :true], fn f -> %>

                            <div class="form-group">
                                <%= text_input f, :name, placeholder: "Name", class: "form-control" %>
                            </div>
                            <div class="form-group">
                                <%= text_input f, :description, placeholder: "Description", class: "form-control" %>
                            </div>

                            <h4 class="card-title">Image Upload</h4>
                            <div class="form-group">
                                <%= file_input f,  :image_files, name: "create_product[images][]", multiple: true, class: "form-control", placeholder: "Upload Image" %>
                            </div>


                            <h4 class="card-title">Video Upload</h4>
                            <div class="form-group">
                                <%= file_input f, :video_files, name: "create_product[videos][]", multiple: true, class: "form-control", placeholder: "Upload Image" %>
                            </div>

                        <br/><br/>
                        <input type="submit" value="Save" class="btn btn-dark mb-2" />
                    <% end %>

I have this three function to filter and process through Guard

def create_product(conn, %{"create_product" => product_params}) when is_list(product_params["images"]) == true and is_list(product_params["videos"]) == true do


def create_product(conn, %{"create_product" => product_params}) when is_list(product_params["images"]) == true and is_list(product_params["videos"]) == false do


  def create_product(conn, %{"create_product" => product_params}) when is_list(product_params["images"]) == false and is_list(product_params["videos"]) == true do

Below is my inspect output

%{
  "description" => "sdfdf",
  "images" => [
    [
      %Plug.Upload{
        content_type: "image/jpeg",
        filename: "deuteronomy-2223-eggstransvestite-transvestism-cross-dressing-cult-prostitutes-lost-and-found-35-638.jpg",
        path: "/tmp/plug-1586/multipart-1586529347-447222130427589-1"
      }
    ]
  ],
  "name" => "Sony"
}

I can see this error is due to accessing product_params["images"].
Can anyone give me insight on how to solve this? Or is there any better way to filter this. Your help is greatly appreciated

Posts: 5

Participants: 2

Read full topic

@neuone wrote:

I’m looking to provide Markdown support for an app, but I want the default markdown to have custom class names and id placed inline. For example:

<h1 class="fw1 headline red">...</h1>

I’m using the library Earmark for the work.
From reading the docs it looks like I can achieve this goal. This is how I set it up, would like some feedback:

defmodule Portal.Blog.Post 

#..omit a lot of code

      defp parse_attr(:body, value) do
        value 
        |> Earmark.as_ast() 
        |> Portal.Blog.Parser.parsing() 
        |> Earmark.as_html!() 
      end

end

I have a module that will parse the body of a post
• Take the value from the body of a blog post
• Pass it to Earmark.as_ast, that returns a list where each item is an HTML node (See Floki)

{tag_name, attributes, children_nodes}
# example
{"p", [{"class", "headline"}], ["Floki"]}

The results from Earmark.as_ast is an {:ok, results, options} and we pass that
to my custom parser.

Portal.Blog.Parser.parsing() will take the list of results:
• map over each item
• pattern match using parse()
• In this example I’m only interested in p, h3 and h1 tags
• every other tag gets a pass through
• Then after each parse() its passed into Earmark.Transform.transform() which convert it into a string
• Then take the list of strings and concat into one

defmodule Portal.Blog.Parser do
    
    def parsing({:ok, results, _option}) do
      Enum.map(results, fn(item) -> 
         parse(item)
         |> Earmark.Transform.transform() 
      end)
      |> Enum.join("")
    end

    # This follows a similar structure to Floki library
    # link here
    # {tag_name, attributes, children_nodes}
    def parse({"p", _attributes, children_nodes }) do
        {"p", [{"class", "fw5 blue"}], children_nodes }
    end

    def parse({"h3", _attributes, children_nodes }) do
        {"h3", [{"class", "f1 fw1 lh-copy"}], children_nodes }
    end

    def parse({"h1", _attributes, children_nodes }) do
      {"h1", [{"class", "fw6"}], children_nodes }
    end

    def parse({"blockquote", _attributes, children_nodes }) do
      {"blockquote", [{"class", "bg-silver"}], children_nodes }
    end

    def parse(item) do
        item
    end

end

Back to my defmodule Portal.Blog.Post:
We then pass the results of Portal.Blog.Parser.parsing() to Earmark.as_html!()

|> Portal.Blog.Parser.parsing() 
|> Earmark.as_html!()

Everything works.
And I think this is how to solve it.
My h1, h3 and p tags all have the correct class names.

The only issue I get is warnings like this:

<no file>:14: warning: Failed to find closing <pre>
<no file>:29: warning: Failed to find closing <pre>
<no file>:1: warning: Failed to find closing <p>
<no file>:15: warning: Failed to find closing <pre>
<no file>:1: warning: Failed to find closing <p>

Looking for feedback if this is how someone would approach this, or is there a better way of solving this.

Thanks

Posts: 1

Participants: 1

Read full topic

@dsd wrote:

I am having trouble getting Erlang port to work on Windows with nouse_stdio option. Wondering whether I can get some guidance on getting that to work. Currently, I have it working without nouse_stdio option using “cmd.exe /c executable”.

Posts: 1

Participants: 1

Read full topic

@david234 wrote:

I am doing images upload.

from form, I am getting these parameters in my controller.

%{
  "description" => "Sample Product",
  "images" => [
    [
      %Plug.Upload{
        content_type: "image/jpeg",
        filename: "deuteronomy-2223-eggstransvestite-transvestism-cross-dressing-cult-prostitutes-lost-and-found-35-638.jpg",
        path: "/tmp/plug-1586/multipart-1586544497-662772098094503-1"
      }
    ],
    [
      %Plug.Upload{
        content_type: "image/png",
        filename: "Screenshot from 2020-01-23 12-53-10.png",
        path: "/tmp/plug-1586/multipart-1586544498-689460437530082-1"
      }
    ],
    [
      %Plug.Upload{
        content_type: "image/png",
        filename: "Screenshot from 2020-01-23 12-52-42.png",
        path: "/tmp/plug-1586/multipart-1586544498-797645038247073-1"
      }
    ]
  ],
  "name" => "TV"
}

Before Repo.insert operation. I need to upload the images to s3 and form a struct like below

Product Table Struct 

%{ name: "TV", description: "Sample Product"}


Product Images Table Struct

[
	%{ product_id: "#{from above inserted result}", image_name: "s3_url_1"},
	%{ product_id: "#{from above inserted result}", image_name: "s3_url_2"},
	%{ product_id: "#{from above inserted result}", image_name: "s3_url_3"}
]

Form Params

%{
  "description" => "Sample Product",
  "images" => [
    [
      %Plug.Upload{
        content_type: "image/jpeg",
        filename: "found-35-638.jpg",
        path: "/tmp/plug-1586/multipart-1586545083-172908842785287-1"
      }
    ],
    [
      %Plug.Upload{
        content_type: "image/png",
        filename: "Screenshot from 2020-01-23 12-53-10.png",
        path: "/tmp/plug-1586/multipart-1586545084-376423312372147-1"
      }
    ],
    [
      %Plug.Upload{
        content_type: "image/png",
        filename: "Screenshot from 2020-01-23 12-52-42.png",
        path: "/tmp/plug-1586/multipart-1586545084-229324763820405-1"
      }
    ]
  ],
  "name" => "TV"
}

I can easily get the product struct. For getting product image table struct first i have to upload all the images and then form a struct.

My question is

1. when I upload images in the same order in which i am getting from form params. For ex
   “images” => [ “image_1”, “image_2”, “image_3”]. So i need to upload all the three images and store in DB in the same order.
   For ex
   Repo.insert( %{image_name: “image_1”})
   Repo.insert( %{image_name: “image_2”})
   Repo.insert( %{image_name: “image_3”})

I mean the order must not change while insert in DB. Because if image 1 is a very big one and it will take time so before image 1 insert image 2 and image 3 will be inserted. That should not happen.

    images_map      = Enum.reduce(images, [], fn x, acc ->

                          image_url = upload_to_s3(x)   #return s3 URL

                          [{:image_name, image_url} | acc]
                      end)

I am getting this output

[image_name: "s3_url", image_name: "s3_url", image_name: "s3_url"]

but I need in this format

[
	%{ image_name: "s3_url_1"},
	%{ image_name: "s3_url_2"},
	%{ image_name: "s3_url_3"}
]

Can anyone give me insight on how to achieve this? Your help is greatly appreciated.Thanks

Posts: 4

Participants: 2

Read full topic

@petelacey wrote:

I have a table in Postgres holding file uploads. I also have a requirement to download a subset of these files as a zip file to the user.

In order to keep a handle on memory and disk IO, I have managed to stream this content straight from the DB to the client. What I have is code that uses Ecto.stream to pluck 10 rows at a time from the DB, send the relevant data to Zap (a streaming zipfile builder), which is then chunked out to the client. All very neat and it works.

def send_zip_contents(id, chunker) do
  query = << an ecto query returning file metadata and contents >>

  one_mb = 1024 * 1024

  Repo.transaction fn ->
    query
    |> Repo.stream(max_rows: 10)
    |> Stream.map(fn upload -> {upload.name, upload.contents} end) 
    |> Zap.into_stream(one_mb)
    |> chunker.()
  end
end

Unfortunately, I need to add one extra file to that zip archive. A summary file that’s not in the DB. My question is this: Is there a way to add one more Zap Entry tuple, i.e. {somefile.name, somefile.contents}, into that stream without breaking it? Some way, for instance, of having Stream.map produce one extra element?

Posts: 3

Participants: 2

Read full topic

@timbot wrote:

Hi there,

I am currently learning Elixir by going through the Mix and OTP guide on elixir-lang.org. I have read through the Getting Started guide already.

I am having trouble getting my head around the use of placeholder arguments when capturing and partially applying named functions. The specific examples that are confusing me are on this page here https://elixir-lang.org/getting-started/mix-otp/agent.html

def get(bucket, key) do
Agent.get(bucket, &Map.get(&1, key))
end

def put(bucket, key, value) do
Agent.update(bucket, &Map.put(&1, key, value))
end

If I understand correctly, the ‘&’ in ‘&Map.get’ and ‘&Map.put’ is capturing the named functions so that they can be passed as arguments to the ‘Agent.get’ and ‘Agent.update’ functions. This makes sense to me. But the ‘&1’ is causing confusion.

I think the ‘&1’ is evaluating to ‘bucket’ in each of these examples. I don’t see what else could be happening here. But if this is the case, then why can’t we just write ‘bucket’ instead of ‘&1’? Wouldn’t that be a whole lot clearer?

I guess I could just carry on with the tutorial without properly understanding this point, but it seems like this is a crucial enough concept to ask an embarrassingly noob question on these forums.

Thanks in advance for helping me to understand this.

Posts: 2

Participants: 2

Read full topic