Create data importation process in a background job

Posted on

To perform this process we must add the following gems:

jquery-fileupload-rails jQuery File Upload for Rails 3.1 Asset Pipeline (Rails 3.2 supported)

gem “jquery-fileupload-rails”

Permit create uploader model for the file selection process and indicates what extensions are allowed

gem ‘carrierwave’, ‘~> 1.0’

Roo implements read access for all common spreadsheet in Excelx, OpenOffice and CSV

gem “roo”, “~> 2.7.0”

Add Roo file format Excel xls

gem ‘roo-xls’

Use sidekiq for background jobs

gem ‘sidekiq’ ”

Create some methods in the User controller file:

def importusersnew

def importuserscreate
@userimport =
file: import
status: UserImport.statuses[“incomplete”]

def importuserspoller
@user_import = UserImport.find(params[:id])


def importuserparams


In the file config/routes.rb create some custom routes for the controllers and the import view:

get ‘importusers/new’, to: ‘users#importusersnew’, as: ‘importusersnew’
post ‘import
users’, to: ‘users#importuserscreate’, as: ‘import_users’

get ‘importuserspoller/:id’, to: ‘users#importuserspoller’, as: ‘importuserspoller’

Template the user import view in the folder app/views/user/importusersnew.html.haml (this is in haml code):

/ here show the success or error messages for the file upload /
%span File upload success

/  Shows the file upload button and the text for the file name   /
= link_to "Select File", "#", id: 'upload-launch', class: "btn btn-primary import-form"
%span#file-name No file selected...

/    Form for the import process this is hide    /
  = form_for :import_file, url: import_users_path, multipart: true do |f|
    = f.file_field :import_users, id: 'fileupload'
    = submit_tag  "file_import", id: 'import-submit'

/    Bar process did with a div and background color    /

/  Animated gif to show the import process  /
      Importing Data...
      = image_tag "spinner.gif", class: "load-img"

/  here show the success or error messages for the users import process  /
  %span Importing users successfully completed
  %span The user import failed contact the system administrator
    .load-finished  /  Show button to return to the users list  /
      = link_to "Return Users", users_path, class: "btn btn-default btn-lg btn-block"

In this template we make a form to do the file selection process, to give stile to the form must hide the form and template a button with a bootstrap and the name file in a text label and use javascript for execute the process

/ CSS for the userimportnew.html.haml file //

upload-users-new {

width: 50%;
margin: auto;
margin-top: 10px;
padding: 0px 5px;
border: 1px solid #ccc;
border-radius: 10px;

#upload-success {
margin: 15px 15px 0px 15px;
padding-left: 20px;
display: none;
#upload-errors {
margin: 15px 15px 0px 15px;
padding-left: 20px;
display: none;

.import-form {
margin: 15px 15px;
.file-name {
display: inline-block;
padding: 15px 15px;

#progress {
margin: 0px 15px 15px;
.bar {
width: 100%;
height: 18px;
background: #e9e6e6;
border-radius: 5px;
position: relative;
-webkit-box-shadow: inset 0 1px 2px rgba(0,0,0,.1);
box-shadow: inset 0 1px 2px rgba(0,0,0,.1);
.bar-success {
width: 0%;
height: 18px;
background: #57b7d4;
border-radius: 5px;
position: absolute;

#import-process {
width: 259px;
margin: auto;
display: none;
.load-text {
font-size: 26px;
font-family: cursive;
display: inline-block;
margin-bottom: 15px;
color: #a2a1a1;
.load-img {
width: 30px;
height: 30px;
display: inline-block;

.sides-margin {
margin: 0px 15px 15px;

#import-success {
display: none;

#import-errors {
display: none;

#import-finished {
display: none;
.load-finished {
margin: 15px 15px;
margin-bottom: 15px;


Javascript code for the import process (the code is on coffescript):

$(‘#upload-launch’).click ->
$(‘#upload-errors’).css(“display”, ‘none’)
$(‘#upload-success’).css(“display”, ‘none’)
$(‘#import-errors’).css(“display”, ‘none’)
$(‘#import-finished’).css(“display”, ‘none’)
dataType: ‘json’
progressall: (e, data) ->
progress = parseInt(data.loaded / * 100, 10)
$(‘#progress .bar-success’).css ‘width’, progress + ‘%’
done: (e, data) ->
if data.result.errors
$(‘#upload-errors’).css(“display”, ‘inherit’)
i = 0
while i < data.result.errors.length
$(‘#upload-errors’).append(“<span>” + data.result.errors[i] + “</span></br>”);
$(‘#upload-success’).css(“display”, ‘inherit’)
$(‘#progress’).css(“display”, ‘none’)
$(‘#import-process’).css(“display”, ‘inherit’)
setInterval (->
url: “/admin/importuserspoller/” +,
dataType: “json”
success: (ajaxdata, textStatus, jqXHR) ->
if (ajax
data.status == “complete”)
$(‘#import-process’).css(“display”, ‘none’)
$(‘#import-success’).css(“display”, ‘inherit’)
$(‘#import-finished’).css(“display”, ‘inherit’)
if (ajax_data.status == “failed”)
$(‘#import-process’).css(“display”, ‘none’)
$(‘#import-errors’).css(“display”, ‘inherit’)
$(‘#import-finished’).css(“display”, ‘inherit’)
), 5000


Generate Uploader model with CarryWave, and add in the white list which kind of files are permit:

class UsersImportUploader < CarrierWave::Uploader::Base
def extension_whitelist
%w(xls xlsx ods)


Create a new class to make all the import process, in the model folder:

class UserImportManager
attraccessor :userimport

def initialize(userimportid)
@userimport = UserImport.find(userimport_id)

# Import data from the file to database
def import
spreadsheet = openspreadsheet
header = spreadsheet.row(1)
row).each do |i|
row = Hash[[header, spreadsheet.row(i)].transpose]
@user =!
@userimport.updateattributes(status: UserImport.statuses[“complete”])

# Open import file and detect extension file
def openspreadsheet
case File.extname(@user
when “.csv” then, password: nil)
when “.ods” then
import.file.path, password: nil)
when “.xls” then, password: nil)
when “.xlsx” then
import.file.path, password: nil)
else raise “Unknown file type: #{file.original_filename}”


In the app folder we create a folder called workers where we will create a class where the import process will be sent to the background job:

class BackgroundImporter
include Sidekiq::Worker

def perform(userimportid)
User.transaction do
@import =
rescue Exception => exeptionmsgs
import = UserImport.find(userimportid)
@userimport.updateattributes(status: UserImport.statuses[“failed”])

Leave a Reply

Your email address will not be published.