Monday, February 12, 2018

HAProxy & Lua: Fifo and asynchronous actions

In some cases, it will be great to process asynchronously some task. Maybe our email server is very slow and you dont want to add this server processing time to the response time of the HTTP request.

The solution is provided in two parts:

  • A Lua FIFO
  • An asychronous action action using tasks and the FIFO.

The FIFO

The fifo is a simple script inspired from the "PIL" manual. There is the library:
-- This library provides fifo functions
--
-- Usage:
--
--   fifo = Fifo.new()
--   fifo:push(data)
--   daa = fifo:pop()

Fifo = {}
Fifo.meta = {}
Fifo.meta.__index = {}
Fifo.new = function()
        local fifo = {}
        fifo.first = 1 -- Always the first data
        fifo.last = 1 -- Alway the last available + 1
        fifo.data = {}
        setmetatable(fifo, Fifo.meta)
        return fifo
end
Fifo.meta.__index.push = function(fifo, data)
        fifo.data[fifo.last] = data
        fifo.last = fifo.last + 1
end
Fifo.meta.__index.pop = function(fifo, data)
        if fifo.first == fifo.last then
                return nil
        end
        local data = fifo.data[fifo.first]
        fifo.data[fifo.first] = nil
        fifo.first = fifo.first + 1
        if fifo.first == fifo.last then
                fifo.first = 1
                fifo.last = 1
        end
        return data
end
The usage of this library is easy. Create new Fifo object. Push and pop elements. Like this
require("fifo")
fifo = Fifo.new()
fifo:push("a")
fifo:push("b")
fifo:push("c")
print(fifo:pop())
print(fifo:pop())
print(fifo:pop())
print(fifo:pop())
This Little sample displays:
$ lua ./test.lua
a
b
c
nil

Asynchronous tasks with Lua & HAProxy

Now, we will use this library to stack task executed asynchronously. We use:

  • A FIFO for storing tasks
  • An HAProxy Lua task for executing it
  • The email library to send email
Fisrt step is initializing the FIFO. The FIFO is initialized in the main part of the Lua file. It it just:
require("fifo")

fifo_email = Fifo.new()
The second step is creation an action which send emails. This action put data where sent to the user. In this example, data is a copy of received request.
core.register_action("async_send_email", { "tcp-req", "http-req", "tcp-res", "http-res" }, function(txn)
   fifo_email:push("Request received:\n" .. txn.req:dup())
end)
And now, the task function which effectively send the data. This function pop the FIFO each seconds looking for jobs.
core.register_task(function()
   local ret
   local reason
   local server = "127.0.0.1"
   local port = 25
   local domain = "arpalert.org"
   local from = "haproxy@arpalert.org"
   local to = "admin@arpalert.org"

   while true do
      -- Process queue
      local data = fifo_email:pop()
      if data == nil then
         core.sleep(1)
      else
         -- Execute action
         local msg = "From: " .. from .. "\r\n" ..
                "To: " .. to .. "\r\n" ..
                "Subject: test - " .. os.date() .. "\r\n" ..
                "\r\n" ..
                data .. "\r\n"
         ret, reason = smtp_send_email(server, port, domain, from, to, msg);
         if ret == false then
            txn:Warning("Can't send email: " .. reason)
         end
      end
   end
end)
Finaly a little bit of haproxy configuration:
global
   lua-load samples.lua
   stats socket /tmp/haproxy.sock mode 644 level admin

defaults
   timeout client 1m
   timeout server 1m

listen sample4
   mode http
   bind *:10040
   http-request lua.async_send_email
   http-request redirect location /ok
That's all

2 comments:

  1. They found that loot boxes 1xbet korea not only shared ‘important structural and psychological similarities with gambling’, however that ‘100% allow for underage gamers to have interaction with these systems'. Indeed, Drummond and Sauer concluded that the presence of loot boxes in video games may subsequently be forming a ‘ripe breeding ground’ for the development of problem gambling among youngsters. Gambling usually preys on essentially the most vulnerable people who search ‘quick and easy money’ to alleviate their issues. Statistics point out that poor and working class people and households are disproportionately affected by the gambling trade . They could not gamble extra frequently than the wealthy, however the costs they pay are higher.

    ReplyDelete
  2. That it combined pot metal and worse in with good metal with by no means a care was a cold thought. MatWeb's database is comprised primarily of data sheets and spec sheets provided by manufacturers and distributors - let them know that you simply noticed their materials knowledge on MatWeb. After you promote or give away a vehicle, you can to|you possibly can} transfer the registration Sweater Vests and vehicle plates to another vehicle you own.

    ReplyDelete