Method: ActiveRecord::Base.import

Defined in:
lib/activerecord-import/import.rb

.import(*args) ⇒ Object

Imports a collection of values to the database.

This is more efficient than using ActiveRecord::Base#create or ActiveRecord::Base#save multiple times. This method works well if you want to create more than one record at a time and do not care about having ActiveRecord objects returned for each record inserted.

This can be used with or without validations. It does not utilize the ActiveRecord::Callbacks during creation/modification while performing the import.

Usage

Model.import array_of_models
Model.import column_names, array_of_values
Model.import column_names, array_of_values, options

Model.import array_of_models

With this form you can call import passing in an array of model objects that you want updated.

Model.import column_names, array_of_values

The first parameter column_names is an array of symbols or strings which specify the columns that you want to update.

The second parameter, array_of_values, is an array of arrays. Each subarray is a single set of values for a new record. The order of values in each subarray should match up to the order of the column_names.

Model.import column_names, array_of_values, options

The first two parameters are the same as the above form. The third parameter, options, is a hash. This is optional. Please see below for what options are available.

Options

  • validate – true|false, tells import whether or not to use \
    ActiveRecord validations. Validations are enforced by default.
  • on_duplicate_key_update – an Array or Hash, tells import to \
    use MySQL's ON DUPLICATE KEY UPDATE ability. See On Duplicate\
    Key Update below.
  • synchronize – an array of ActiveRecord instances for the model that you are currently importing data into. This synchronizes existing model instances in memory with updates from the import.
  • timestamps – true|false, tells import to not add timestamps \ (if false) even if record timestamps is disabled in ActiveRecord::Base
  • +recursive – true|false, tells import to import all autosave association if the adapter supports setting the primary keys of the newly imported objects.

Arraying your arguments – Ruby

The list of parameters passed to an object is, in fact, available as a list. To do this, we use what is called the splat operator – which is just an asterisk (*).

The splat operator is used to handle methods which have a variable parameter list. Let’s use it to create an add method that can handle any number of parameters.

We use the inject method to iterate over arguments, which is covered in the chapter on Collections. It isn’t directly relevant to this lesson, but do look it up if it piques your interest.

Example Code:

def add(*numbers)
  numbers.inject(0) { |sum, number| sum + number }
end

puts add(1)
puts add(1, 2)
puts add(1, 2, 3)
puts add(1, 2, 3, 4)

The splat operator works both ways – you can use it to convert arrays to parameter lists as easily as we just converted a parameter list to an array.

Inject in Ruby

The syntax for the inject method is as follows:

inject (value_initial) { |result_memo, object| block }

Let’s solve the above example i.e.

[1, 2, 3, 4].inject(0) { |result, element| result + element }

which gives the 10 as the output.

So, before starting let’s see what are the values stored in each variables:

result = 0 The zero came from inject(value) which is 0

element = 1 It is first element of the array.

Okey!!! So, let’s start understanding the above example

Step :1 [1, 2, 3, 4].inject(0) { |0, 1| 0 + 1 }

Step :2 [1, 2, 3, 4].inject(0) { |1, 2| 1 + 2 }

Step :3 [1, 2, 3, 4].inject(0) { |3, 3| 3 + 3 }

Step :4 [1, 2, 3, 4].inject(0) { |6, 4| 6 + 4 }

Step :5 [1, 2, 3, 4].inject(0) { |10, Now no elements left in the array, so it'll return 10 from this step| }

Here Bold-Italic values are elements fetch from array and the simply Bold values are the resultant values.

I hope that you understand the working of the #inject method of the #ruby.

 

Mongo::Error::OperationFailure: Cursor not found

Lately, I’ve been running into this error while running my nightly automation scripts. From my experience in resolving nagging errors, this one too was another of those annoying/inconsistent errors which did not have any concrete solutions on the internet.  This post is for the benefit of those fortunate people(unlike me) who will encounter this error in the future.

Small Background on the task:I had to query all data which was in my MongoDB server one-by-one and compare it on real-time with my api responses. I am using the ‘mongo’ ruby driver gem to interact with the db.The total data in the db was ~3.5 lac records but while running the script – at around the ~350 iteration mark I was getting this error :-

Mongo::Error::OperationFailure:
  Cursor not found, cursor id: 79727049273 (43)
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/operation/result.rb:256:in `validate!'
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/operation/executable.rb:37:in `block in execute'
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/server/connection_pool.rb:107:in `with_connection'
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/server.rb:242:in `with_connection'
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/operation/executable.rb:35:in `execute'
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/cursor.rb:188:in `block in get_more'
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/retryable.rb:51:in `read_with_retry'
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/cursor.rb:187:in `get_more'
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/cursor.rb:113:in `each'
/home/qaserver/.rvm/gems/ruby-2.3.0/gems/mongo-2.4.0/lib/mongo/collection/view/iterable.rb:44:in `each'
./spec/all_usecases_spec/rovi_ott_validation_spec/rovi_ott_links_validation_for_all_programs_spec.rb:66:in `block (2 levels) in '

div>

254      # @since 2.0.0
255      def validate!
256        !successful? ? raise(Error::OperationFailure.new(parser.message)) : self
257      end
258
259# Install the coderay gem to get syntax highlightingpre>

The crazy part was this error was not at all consistent but would happen at times. I would overlook this by re-running  my scripts.One day suddenly out of nowhere this error became almost 100% consistent! On priority, I had to find a solution for it.

After doing some amount of reading, I realised that this error is to do with the cursor which gets created while querying the db. So what happens is MongoDB returns a cursor when the query happens. In my case as my query is one which ‘finds all’ I do not fully know if multiple cursors are returned for each sub-query or a single cursor is returned which loops through the whole db. I need some more clarification on that.

But what I understand is that MongoDB closes all cursors that have been inactive for 10 minutes.It has something called a cursor timeout to do the same. So maybe one such cursor created was getting inactive after a particular time.

On more exploring I understood that there is a way to disable this cursor timeout. The hard part was to find the key word for this cursor timeout for the ruby driver which I was using, in my case the ruby driver ‘mongo’. Going through multiple stackoverflows which gave some incorrect solutions like use ‘:timeout => false’ I had to struggle my way to find this answer.

After going through the Mongo Ruby Driver documentation(which has a very confusing sequence) thouroghly, I found my answer!

There is an option while querying called ‘no_cursor_timeout’ which must be used to disable this cursor timeout. Here’s how you implement it :-

coll.find({:date => { ‘$eq’ => Date.today }}).no_cursor_timeout.each do |doc|

          ########## Code goes in here ###########

end

Active Record Basics

1 What is Active Record?

Active Record is the M in MVC – the model – which is the layer of the system responsible for representing business data and logic. Active Record facilitates the creation and use of business objects whose data requires persistent storage to a database. It is an implementation of the Active Record pattern which itself is a description of an Object Relational Mapping system.

1.1 The Active Record Pattern

Active Record was described by Martin Fowler in his book Patterns of Enterprise Application Architecture. In Active Record, objects carry both persistent data and behavior which operates on that data. Active Record takes the opinion that ensuring data access logic as part of the object will educate users of that object on how to write to and read from the database.

1.2 Object Relational Mapping

Object Relational Mapping, commonly referred to as its abbreviation ORM, is a technique that connects the rich objects of an application to tables in a relational database management system. Using ORM, the properties and relationships of the objects in an application can be easily stored and retrieved from a database without writing SQL statements directly and with less overall database access code.

1.3 Active Record as an ORM Framework

Active Record gives us several mechanisms, the most important being the ability to:

  • Represent models and their data.
  • Represent associations between these models.
  • Represent inheritance hierarchies through related models.
  • Validate models before they get persisted to the database.
  • Perform database operations in an object-oriented fashion.

2 Convention over Configuration in Active Record

When writing applications using other programming languages or frameworks, it may be necessary to write a lot of configuration code. This is particularly true for ORM frameworks in general. However, if you follow the conventions adopted by Rails, you’ll need to write very little configuration (in some cases no configuration at all) when creating Active Record models. The idea is that if you configure your applications in the very same way most of the time then this should be the default way. Thus, explicit configuration would be needed only in those cases where you can’t follow the standard convention.

2.1 Naming Conventions

By default, Active Record uses some naming conventions to find out how the mapping between models and database tables should be created. Rails will pluralize your class names to find the respective database table. So, for a class Book, you should have a database table called books. The Rails pluralization mechanisms are very powerful, being capable of pluralizing (and singularizing) both regular and irregular words. When using class names composed of two or more words, the model class name should follow the Ruby conventions, using the CamelCase form, while the table name must contain the words separated by underscores. Examples:

  • Database Table – Plural with underscores separating words (e.g., book_clubs).
  • Model Class – Singular with the first letter of each word capitalized (e.g., BookClub).
Model / Class Table / Schema
Article articles
LineItem line_items
Deer deers
Mouse mice
Person people

2.2 Schema Conventions

Active Record uses naming conventions for the columns in database tables, depending on the purpose of these columns.

  • Foreign keys – These fields should be named following the pattern singularized_table_name_id (e.g., item_id, order_id). These are the fields that Active Record will look for when you create associations between your models.
  • Primary keys – By default, Active Record will use an integer column named id as the table’s primary key. When using Active Record Migrations to create your tables, this column will be automatically created.

There are also some optional column names that will add additional features to Active Record instances:

  • created_at – Automatically gets set to the current date and time when the record is first created.
  • updated_at – Automatically gets set to the current date and time whenever the record is updated.
  • lock_version – Adds optimistic locking to a model.
  • type – Specifies that the model uses Single Table Inheritance.
  • (association_name)_type – Stores the type for polymorphic associations.
  • (table_name)_count – Used to cache the number of belonging objects on associations. For example, a comments_count column in an Article class that has many instances of Comment will cache the number of existent comments for each article.

While these column names are optional, they are in fact reserved by Active Record. Steer clear of reserved keywords unless you want the extra functionality. For example, type is a reserved keyword used to designate a table using Single Table Inheritance (STI). If you are not using STI, try an analogous keyword like “context”, that may still accurately describe the data you are modeling.

3 Creating Active Record Models

It is very easy to create Active Record models. All you have to do is to subclass the ApplicationRecord class and you’re good to go:

class Product < ApplicationRecord
end

This will create a Product model, mapped to a products table at the database. By doing this you’ll also have the ability to map the columns of each row in that table with the attributes of the instances of your model. Suppose that the products table was created using an SQL statement like:

CREATE TABLE products (
   id int(11) NOT NULL auto_increment,
   name varchar(255),
   PRIMARY KEY  (id)
);

Following the table schema above, you would be able to write code like the following:

p = Product.new
p.name = "Some Book"
puts p.name # "Some Book"

4 Overriding the Naming Conventions

What if you need to follow a different naming convention or need to use your Rails application with a legacy database? No problem, you can easily override the default conventions.

ApplicationRecord inherits from ActiveRecord::Base, which defines a number of helpful methods. You can use the ActiveRecord::Base.table_name= method to specify the table name that should be used:

class Product < ApplicationRecord
  self.table_name = "my_products"
end

If you do so, you will have to define manually the class name that is hosting the fixtures (my_products.yml) using the set_fixture_class method in your test definition:

class ProductTest < ActiveSupport::TestCase
  set_fixture_class my_products: Product
  fixtures :my_products
  ...
end

It’s also possible to override the column that should be used as the table’s primary key using the ActiveRecord::Base.primary_key= method:

class Product < ApplicationRecord
  self.primary_key = "product_id"
end

5 CRUD: Reading and Writing Data

CRUD is an acronym for the four verbs we use to operate on data: Create, Read, Update and Delete. Active Record automatically creates methods to allow an application to read and manipulate data stored within its tables.

5.1 Create

Active Record objects can be created from a hash, a block or have their attributes manually set after creation. The new method will return a new object while create will return the object and save it to the database.

For example, given a model User with attributes of name and occupation, the create method call will create and save a new record into the database:

user = User.create(name: "David", occupation: "Code Artist")

Using the new method, an object can be instantiated without being saved:

user = User.new
user.name = "David"
user.occupation = "Code Artist"

A call to user.save will commit the record to the database.

Finally, if a block is provided, both create and new will yield the new object to that block for initialization:

user = User.new do |u|
  u.name = "David"
  u.occupation = "Code Artist"
end

5.2 Read

Active Record provides a rich API for accessing data within a database. Below are a few examples of different data access methods provided by Active Record.

# return a collection with all users
users = User.all
# return the first user
user = User.first
# return the first user named David
david = User.find_by(name: 'David')
# find all users named David who are Code Artists and sort by created_at in reverse chronological order
users = User.where(name: 'David', occupation: 'Code Artist').order(created_at: :desc)

You can learn more about querying an Active Record model in the Active Record Query Interfaceguide.

5.3 Update

Once an Active Record object has been retrieved, its attributes can be modified and it can be saved to the database.

user = User.find_by(name: 'David')
user.name = 'Dave'
user.save

A shorthand for this is to use a hash mapping attribute names to the desired value, like so:

user = User.find_by(name: 'David')
user.update(name: 'Dave')

This is most useful when updating several attributes at once. If, on the other hand, you’d like to update several records in bulk, you may find the update_all class method useful:

User.update_all "max_login_attempts = 3, must_change_password = 'true'"

5.4 Delete

Likewise, once retrieved an Active Record object can be destroyed which removes it from the database.

user = User.find_by(name: 'David')
user.destroy

6 Validations

Active Record allows you to validate the state of a model before it gets written into the database. There are several methods that you can use to check your models and validate that an attribute value is not empty, is unique and not already in the database, follows a specific format and many more.

Validation is a very important issue to consider when persisting to the database, so the methods save and update take it into account when running: they return false when validation fails and they didn’t actually perform any operation on the database. All of these have a bang counterpart (that is, save! and update!), which are stricter in that they raise the exception ActiveRecord::RecordInvalid if validation fails. A quick example to illustrate:

class User < ApplicationRecord
  validates :name, presence: true
end
user = User.new
user.save  # => false
user.save! # => ActiveRecord::RecordInvalid: Validation failed: Name can't be blank

You can learn more about validations in the Active Record Validations guide.

7 Callbacks

Active Record callbacks allow you to attach code to certain events in the life-cycle of your models. This enables you to add behavior to your models by transparently executing code when those events occur, like when you create a new record, update it, destroy it and so on. You can learn more about callbacks in the Active Record Callbacks guide.

8 Migrations

Rails provides a domain-specific language for managing a database schema called migrations. Migrations are stored in files which are executed against any database that Active Record supports using rake. Here’s a migration that creates a table:

class CreatePublications < ActiveRecord::Migration[5.0]
  def change
    create_table :publications do |t|
      t.string :title
      t.text :description
      t.references :publication_type
      t.integer :publisher_id
      t.string :publisher_type
      t.boolean :single_issue
      t.timestamps
    end
    add_index :publications, :publication_type_id
  end
end

Rails keeps track of which files have been committed to the database and provides rollback features. To actually create the table, you’d run rails db:migrate and to roll it back, rails db:rollback.

Note that the above code is database-agnostic: it will run in MySQL, PostgreSQL, Oracle and others. You can learn more about migrations in the Active Record Migrations guide.

Courtesy: guides.rubyonrails.org

Ruby – Read/Write Modes on Files

IO Open Mode

Ruby allows the following open modes:

"r"  Read-only, starts at beginning of file  (default mode).

"r+" Read-write, starts at beginning of file.

"w"  Write-only, truncates existing file
     to zero length or creates a new file for writing.

"w+" Read-write, truncates existing file to zero length
     or creates a new file for reading and writing.

"a"  Write-only, each write call appends data at end of file.
     Creates a new file for writing if file does not exist.

"a+" Read-write, each write call appends data at end of file.
     Creates a new file for reading and writing if file does
     not exist.

The following modes must be used separately, and along with one or more of the modes seen above.

"b"  Binary file mode
     Suppresses EOL  CRLF conversion on Windows. And
     sets external encoding to ASCII-8BIT unless explicitly
     specified.

"t"  Text file mode

When the open mode of original IO is read only, the mode cannot be changed to be writable. Similarly, the open mode cannot be changed from write only to readable.

When such a change is attempted the error is raised in different locations according to the platform.

Querying using mongo ruby driver

Suppose you have a db named ‘bands’ with a collection named ‘metal’, finding the band named ‘Dream Theater’ would be like this :-

client = Mongo::Client.new([ '127.0.0.1:27017' ], :database => 'bands')

client[:metal].find(:name => 'Dream Theater').each do |doc|
  #=> Yields a BSON::Document.
puts doc
end
You can then convert the bson document to json by using ‘to_json’ and proceed with manipulation of the data

 

Ruby Variables

Variables are the memory locations which hold any data to be used by any program.

There are five types of variables supported by Ruby. You already have gone through a small description of these variables in previous chapter as well. These five types of variables are explained in this chapter.

Ruby Global Variables:

Global variables begin with $. Uninitialized global variables have the value nil and produce warnings with the -w option.

Assignment to global variables alters global status. It is not recommended to use global variables. They make programs cryptic.

Here is an example showing usage of global variable.

#!/usr/bin/ruby

$global_variable = 10
class Class1
  def print_global
     puts "Global variable in Class1 is #$global_variable"
  end
end
class Class2
  def print_global
     puts "Global variable in Class2 is #$global_variable"
  end
end

class1obj = Class1.new
class1obj.print_global
class2obj = Class2.new
class2obj.print_global

Here $global_variable is a global variable. This will produce the following result:

NOTE: In Ruby you CAN access value of any variable or constant by putting a hash (#) character just before that variable or constant.

Global variable in Class1 is 10
Global variable in Class2 is 10

Ruby Instance Variables:

Instance variables begin with @. Uninitialized instance variables have the value nil and produce warnings with the -w option.

Here is an example showing usage of Instance Variables.

#!/usr/bin/ruby

class Customer
   def initialize(id, name, addr)
      @cust_id=id
      @cust_name=name
      @cust_addr=addr
   end
   def display_details()
      puts "Customer id #@cust_id"
      puts "Customer name #@cust_name"
      puts "Customer address #@cust_addr"
    end
end

# Create Objects
cust1=Customer.new("1", "John", "Wisdom Apartments, Ludhiya")
cust2=Customer.new("2", "Poul", "New Empire road, Khandala")

# Call Methods
cust1.display_details()
cust2.display_details()

Here, @cust_id, @cust_name and @cust_addr are instance variables. This will produce the following result:

Customer id 1
Customer name John
Customer address Wisdom Apartments, Ludhiya
Customer id 2
Customer name Poul
Customer address New Empire road, Khandala

Ruby Class Variables:

Class variables begin with @@ and must be initialized before they can be used in method definitions.

Referencing an uninitialized class variable produces an error. Class variables are shared among descendants of the class or module in which the class variables are defined.

Overriding class variables produce warnings with the -w option.

Here is an example showing usage of class variable:

#!/usr/bin/ruby

class Customer
   @@no_of_customers=0
   def initialize(id, name, addr)
      @cust_id=id
      @cust_name=name
      @cust_addr=addr
      @@no_of_customers += 1
   end
   def display_details()
      puts "Customer id #@cust_id"
      puts "Customer name #@cust_name"
      puts "Customer address #@cust_addr"
    end
    def total_no_of_customers()
       puts "Total number of customers: #@@no_of_customers"
    end
end

# Create Objects
cust1=Customer.new("1", "John", "Wisdom Apartments, Ludhiya")
cust2=Customer.new("2", "Poul", "New Empire road, Khandala")

# Call Methods
cust1.total_no_of_customers()
cust2.total_no_of_customers()

Here @@no_of_customers is a class variable. This will produce the following result:

Total number of customers: 1
Total number of customers: 2

Ruby Local Variables:

Local variables begin with a lowercase letter or _. The scope of a local variable ranges from class, module, def, or do to the corresponding end or from a block’s opening brace to its close brace {}.

When an uninitialized local variable is referenced, it is interpreted as a call to a method that has no arguments.

Assignment to uninitialized local variables also serves as variable declaration. The variables start to exist until the end of the current scope is reached. The lifetime of local variables is determined when Ruby parses the program.

In the above example local variables are id, name and addr.

Ruby Constants:

Constants begin with an uppercase letter. Constants defined within a class or module can be accessed from within that class or module, and those defined outside a class or module can be accessed globally.

Constants may not be defined within methods. Referencing an uninitialized constant produces an error. Making an assignment to a constant that is already initialized produces a warning.

#!/usr/bin/ruby

class Example
   VAR1 = 100
   VAR2 = 200
   def show
       puts "Value of first Constant is #{VAR1}"
       puts "Value of second Constant is #{VAR2}"
   end
end

# Create Objects
object=Example.new()
object.show

Here VAR1 and VAR2 are constant. This will produce the following result:

Value of first Constant is 100
Value of second Constant is 200

Ruby Pseudo-Variables:

They are special variables that have the appearance of local variables but behave like constants. You can not assign any value to these variables.

  • self: The receiver object of the current method.
  • true: Value representing true.
  • false: Value representing false.
  • nil: Value representing undefined.
  • __FILE__: The name of the current source file.
  • __LINE__: The current line number in the source file.

Courtesy: Tutorials Point