MySandbox: 2013

Friday, November 22, 2013

Using dbca to create a plugable Oracle 12c database

Wednesday, November 20, 2013

Installing Oracle 12c Grid

As use 'oracle' run runInstaller the grid directory created after untaring the grid binaries.

Wednesday, November 13, 2013

File "/usr/bin/yum", line 31 except KeyboardInterrupt, e: ^ SyntaxError: invalid syntax

I was getting the "invalid syntax" error with yum

[root@isvx3 ~]# yum
File "/usr/bin/yum", line 31
except KeyboardInterrupt, e:
^
SyntaxError: invalid syntax
[root@isvx3 ~]#
[root@isvx3 ~]#

I found that the error started after I upgraded the python on the machine to 3.3.2

[root@isvx3 ~]# python -V
Python 3.3.2
[root@isvx3 ~]#

[root@isvx3 ~]# vim $(which yum)
#!/usr/bin/python

import sys
try:
import yum
except ImportError:
print >> sys.stderr, """\
..........................
..........................

The fix for this is to use the older version of python, which in my case was Python 2.4.3

[root@isvx3 ~]# /usr/bin/python2.4 -V
Python 2.4.3

[root@isvx3 ~]# more $(which yum)
#!/usr/bin/python2.4

import sys
try:
import yum
except ImportError:
print >> sys.stderr, """\
.............................
..............................

Friday, November 8, 2013

Importing data into a MongoDB database using Kettle from a csv file

In a previous post we saw how to import data into a MongoDB database from a cvs file using mongoimport. In this post we will import data into the MongoDB database from a cvs file using Kettle.

This youtube video show how to import data from a text file into MongoDB
http://www.youtube.com/watch?v=Tgyrd1UiQhE

The screen below shows the configuration of the CSV input

The screen below show the configuration of the MogoDB output

The below screen shows the transformation between the CVS file input -> MongoDB output was created, and run.

Next we will connect to MongoDB, to see if the data has been indeed imported. Connect to the mongo shell

> use mydbdemo
switched to db mydbdemo
> db.projects.find().toArray()
[
        {
                "_id" : ObjectId("527d4e5e73ca0be1b2bc2504"),
                "ISV Name" : "Oracle",
                "Storage Brand" : "SVC",
                "Project" : "SVC Chubby nodes and Oracle",
                "Owner" : "Mayur"
        },
        {
                "_id" : ObjectId("527d4e5e73ca0be1b2bc2505"),
                "ISV Name" : "Oracle",
                "Storage Brand" : "IBM Tape",
                "Project" : "Oracle backup",
                "Owner" : "Shashank"
        },
        {
                "_id" : ObjectId("527d4e5e73ca0be1b2bc2506"),
                "ISV Name" : "Oracle",
                "Storage Brand" : "V7000",
                "Project" : "Oracle 12c on V7000",
                "Owner" : "Mayur"
        },
        {
                "_id" : ObjectId("527d4e5e73ca0be1b2bc2507"),
                "ISV Name" : "Oracle",
                "Storage Brand" : "DS8870",
                "Project" : "Easy Tier 5 for Oracle",
                "Owner" : "Mayur"
        }
]

We can also import data directly from a Microsoft Excel spreadsheet into MongoDB using Kettle.

Installing Pentaho Kettle on RedHat Linux

I downloaded Pentaho Kettle from http://sourceforge.net/projects/pentaho/files/Data%20Integration/4.4.0-stable/

[root@isvx3 ~]# java -version
java version "1.6.0_31"
Java(TM) SE Runtime Environment (build 1.6.0_31-b04)
Java HotSpot(TM) 64-Bit Server VM (build 20.6-b01, mixed mode)
[root@isvx3 ~]#
[root@isvx3 ~]# cat /proc/version
Linux version 2.6.18-348.16.1.el5 (mockbuild@x86-012.build.bos.redhat.com) (gcc version 4.1.2 20080704 (Red Hat 4.1.2-54)) #1 SMP Sat Jul 27 01:05:23 EDT 2013
[root@isvx3 ~]#
[root@isvx3 ~]# uname -m
x86_64

On untarring pdi-ce-4.4.0-stable.tar it created the data-integration directory

[root@isvx3 Desktop]# cd data-integration/
[root@isvx3 data-integration]# ls
Carte.bat                    libswt
carte.sh                     Pan.bat
Data Integration 32-bit.app pan.sh
Data Integration 64-bit.app plugins
docs                         plugins_old
Encr.bat                     pwd
encr.sh                      README_INFOBRIGHT.txt
generateClusterSchema.sh     README_LINUX.txt
hs_err_pid10080.log          README_OSX.txt
hs_err_pid10405.log          README_UNIX_AS400.txt
hs_err_pid3841.log           run_kettle_cluster_example.bat
hs_err_pid4875.log           runSamples.sh
hs_err_pid7690.log           samples
Import.bat                   set-pentaho-env.bat
import-rules.xml             set-pentaho-env.sh
import.sh                    simple-jndi
Kitchen.bat                  Spoon.bat
kitchen.sh                   spoon.ico
launcher                     spoon.png
lib                          spoon.sh
libext                       ui

On running spoon.sh I got a SIGSEGV

[root@isvx7 data-integration]# ./spoon.sh
/root/Desktop/data-integration
WARN 08-11 11:41:00,110 - Unable to load Hadoop Configuration from "file:///root/Desktop/data-integration/plugins/pentaho-big-data-plugin/hadoop-configurations/mapr". For more information enable debug logging.
INFO 08-11 11:41:00,139 - Spoon - Logging goes to file:///tmp/spoon_5011469f-48a5-11e3-bce7-cf39551697e4.log
#
# A fatal error has been detected by the Java Runtime Environment:
#
# SIGSEGV (0xb) at pc=0x0000000000000000, pid=30616, tid=48006614890816
#
# JRE version: 6.0_22-b04
# Java VM: Java HotSpot(TM) 64-Bit Server VM (17.1-b03 mixed mode linux-amd64 )
# Problematic frame:
# C 0x0000000000000000
#
# An error report file with more information is saved as:
# /root/Desktop/data-integration/hs_err_pid30616.log
#
# If you would like to submit a bug report, please visit:
# http://java.sun.com/webapps/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
./spoon.sh: line 163: 30616 Aborted "$_PENTAHO_JAVA" $OPT $STARTUP -lib $LIBPATH "${1+$@}"
[root@isvx7 data-integration]#

I found the fix to this issue on one of the stackoverflow postings: http://stackoverflow.com/questions/15943531/jdk-fatal-error-when-launching-pentaho-spoon-on-centos

[root@isvx7 data-integration]# export OPT="-Dorg.eclipse.swt.browser.XULRunnerPath=/dev/null"
[root@isvx7 data-integration]# ./spoon.sh

Thursday, November 7, 2013

Connecting to a MongoDB database from a Python script

Now that we have pymongo setup, we will connect to the MongoDB database (mydb) from a python script. The source of the script is from the "MongoDB and Python Patterns and processes for the popular document-oriented database By Niall O'Higgins"

#!/usr/bin/python
import sys
from pymongo import Connection
from pymongo.errors import ConnectionFailure

def main():
    """ Connect to MongoDB """
    try:
        c = Connection(host="localhost", port=27017)
        print ("Connected successfully")
    except ConnectionFailure, e:
        sys.stderr.write("Could not connect to MongoDB: %s" % e)
        sys.exit(1)
    # Get a Database handle to a database named "mydb"
    dbh = c["mydb"]

    # Demonstrate the db.connection property to retrieve a reference to the
    # Connection object should it go out of scope. In most cases, keeping a
    # reference to the Database object for the lifetime of your program should
    # be sufficient.

    assert dbh.connection == c
    print "Successfully set up a database handle"

if __name__ == "__main__":
    main()

Output on executing the script test.py
Here is what I did to install the pymongo module on my system.

[root@isvx7 ~]# ./test.py
Connected successfully
Successfully set up a database handle

Message on the mongod console:

Thu Nov 7 12:48:56 [initandlisten] connection accepted from 127.0.0.1:33804 #4 (1 connection now open)
Thu Nov 7 12:48:56 [conn4] end connection 127.0.0.1:33804 (0 connections now open)

Wednesday, November 6, 2013

Installing pymongo module for Python 2.4.3

Here is what I did to install the pymongo module on my system which had Python 2.4.3

[root@isvx7 Desktop]# python -V
Python 2.4.3

Below is the pymongo installation documentation:
http://api.mongodb.org/python/current/installation.html

I saw that I didn't have pip nor easy_install on my system.

I donloaded pip-1.4.1.tar.gz from https://pypi.python.org/pypi/pip and followed the installation instructions from http://www.pip-installer.org/en/latest/installing.html

and downloaded ez_setup-0.9.tar.gz from https://pypi.python.org/pypi/ez_setup

[root@isvx7 ez_setup-0.9]# pwd
/root/Desktop/ez_setup-0.9
[root@isvx7 ez_setup-0.9]# ls
distribute-0.6.14.tar.gz ez_setup.egg-info MANIFEST.in PKG-INFO setup.cfg
distribute_setup.py ez_setup.py NEWS.txt README.rst setup.py

[root@isvx7 ez_setup-0.9]# python ez_setup.py
Extracting in /tmp/tmp3RVJMo
Now working in /tmp/tmp3RVJMo/distribute-0.6.14
Installing Distribute
Before install bootstrap.
Scanning installed packages
Setuptools installation detected at /usr/lib/python2.4/site-packages
Non-egg installation
Removing elements out of the way...
Already patched.
/usr/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg-info already patched.
running install
install_dir /usr/lib/python2.4/site-packages/
running bdist_egg
running egg_info
writing distribute.egg-info/PKG-INFO
writing top-level names to distribute.egg-info/top_level.txt
writing dependency_links to distribute.egg-info/dependency_links.txt
writing entry points to distribute.egg-info/entry_points.txt
reading manifest file 'distribute.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
writing manifest file 'distribute.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
copying pkg_resources.py -> build/lib
copying easy_install.py -> build/lib
copying site.py -> build/lib
creating build/lib/setuptools
copying setuptools/extension.py -> build/lib/setuptools
copying setuptools/dist.py -> build/lib/setuptools
copying setuptools/package_index.py -> build/lib/setuptools
copying setuptools/archive_util.py -> build/lib/setuptools
copying setuptools/__init__.py -> build/lib/setuptools
copying setuptools/depends.py -> build/lib/setuptools
copying setuptools/sandbox.py -> build/lib/setuptools
creating build/lib/setuptools/tests
copying setuptools/tests/test_packageindex.py -> build/lib/setuptools/tests
copying setuptools/tests/test_sandbox.py -> build/lib/setuptools/tests
copying setuptools/tests/test_develop.py -> build/lib/setuptools/tests
copying setuptools/tests/test_build_ext.py -> build/lib/setuptools/tests
copying setuptools/tests/test_resources.py -> build/lib/setuptools/tests
copying setuptools/tests/test_upload_docs.py -> build/lib/setuptools/tests
copying setuptools/tests/test_easy_install.py -> build/lib/setuptools/tests
copying setuptools/tests/server.py -> build/lib/setuptools/tests
copying setuptools/tests/doctest.py -> build/lib/setuptools/tests
copying setuptools/tests/__init__.py -> build/lib/setuptools/tests
creating build/lib/setuptools/command
copying setuptools/command/setopt.py -> build/lib/setuptools/command
copying setuptools/command/alias.py -> build/lib/setuptools/command
copying setuptools/command/saveopts.py -> build/lib/setuptools/command
copying setuptools/command/register.py -> build/lib/setuptools/command
copying setuptools/command/develop.py -> build/lib/setuptools/command
copying setuptools/command/bdist_wininst.py -> build/lib/setuptools/command
copying setuptools/command/upload.py -> build/lib/setuptools/command
copying setuptools/command/install_lib.py -> build/lib/setuptools/command
copying setuptools/command/bdist_egg.py -> build/lib/setuptools/command
copying setuptools/command/install_scripts.py -> build/lib/setuptools/command
copying setuptools/command/sdist.py -> build/lib/setuptools/command
copying setuptools/command/test.py -> build/lib/setuptools/command
copying setuptools/command/egg_info.py -> build/lib/setuptools/command
copying setuptools/command/upload_docs.py -> build/lib/setuptools/command
copying setuptools/command/bdist_rpm.py -> build/lib/setuptools/command
copying setuptools/command/install.py -> build/lib/setuptools/command
copying setuptools/command/rotate.py -> build/lib/setuptools/command
copying setuptools/command/build_py.py -> build/lib/setuptools/command
copying setuptools/command/easy_install.py -> build/lib/setuptools/command
copying setuptools/command/install_egg_info.py -> build/lib/setuptools/command
copying setuptools/command/build_ext.py -> build/lib/setuptools/command
copying setuptools/command/__init__.py -> build/lib/setuptools/command
copying setuptools/cli.exe -> build/lib/setuptools
copying setuptools/gui.exe -> build/lib/setuptools
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/setuptools
copying build/lib/setuptools/extension.py -> build/bdist.linux-x86_64/egg/setuptools
copying build/lib/setuptools/cli.exe -> build/bdist.linux-x86_64/egg/setuptools
copying build/lib/setuptools/dist.py -> build/bdist.linux-x86_64/egg/setuptools
copying build/lib/setuptools/gui.exe -> build/bdist.linux-x86_64/egg/setuptools
copying build/lib/setuptools/package_index.py -> build/bdist.linux-x86_64/egg/setuptools
copying build/lib/setuptools/archive_util.py -> build/bdist.linux-x86_64/egg/setuptools
creating build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/test_packageindex.py -> build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/test_sandbox.py -> build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/test_develop.py -> build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/test_build_ext.py -> build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/test_resources.py -> build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/test_upload_docs.py -> build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/test_easy_install.py -> build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/server.py -> build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/doctest.py -> build/bdist.linux-x86_64/egg/setuptools/tests
copying build/lib/setuptools/tests/__init__.py -> build/bdist.linux-x86_64/egg/setuptools/tests
creating build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/setopt.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/alias.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/saveopts.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/register.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/develop.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/bdist_wininst.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/upload.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/install_lib.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/bdist_egg.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/install_scripts.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/sdist.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/test.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/egg_info.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/upload_docs.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/bdist_rpm.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/install.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/rotate.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/build_py.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/easy_install.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/install_egg_info.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/build_ext.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/command/__init__.py -> build/bdist.linux-x86_64/egg/setuptools/command
copying build/lib/setuptools/__init__.py -> build/bdist.linux-x86_64/egg/setuptools
copying build/lib/setuptools/depends.py -> build/bdist.linux-x86_64/egg/setuptools
copying build/lib/setuptools/sandbox.py -> build/bdist.linux-x86_64/egg/setuptools
copying build/lib/pkg_resources.py -> build/bdist.linux-x86_64/egg
copying build/lib/site.py -> build/bdist.linux-x86_64/egg
copying build/lib/easy_install.py -> build/bdist.linux-x86_64/egg
byte-compiling build/bdist.linux-x86_64/egg/setuptools/extension.py to extension.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/dist.py to dist.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/package_index.py to package_index.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/archive_util.py to archive_util.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/test_packageindex.py to test_packageindex.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/test_sandbox.py to test_sandbox.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/test_develop.py to test_develop.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/test_build_ext.py to test_build_ext.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/test_resources.py to test_resources.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/test_upload_docs.py to test_upload_docs.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/test_easy_install.py to test_easy_install.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/server.py to server.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/doctest.py to doctest.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/tests/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/setopt.py to setopt.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/alias.py to alias.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/saveopts.py to saveopts.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/register.py to register.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/develop.py to develop.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/bdist_wininst.py to bdist_wininst.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/upload.py to upload.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install_lib.py to install_lib.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py to bdist_egg.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install_scripts.py to install_scripts.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/sdist.py to sdist.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/test.py to test.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py to egg_info.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/upload_docs.py to upload_docs.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/bdist_rpm.py to bdist_rpm.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install.py to install.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/rotate.py to rotate.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/build_py.py to build_py.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py to easy_install.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install_egg_info.py to install_egg_info.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/build_ext.py to build_ext.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/depends.py to depends.pyc
byte-compiling build/bdist.linux-x86_64/egg/setuptools/sandbox.py to sandbox.pyc
byte-compiling build/bdist.linux-x86_64/egg/pkg_resources.py to pkg_resources.pyc
byte-compiling build/bdist.linux-x86_64/egg/site.py to site.pyc
byte-compiling build/bdist.linux-x86_64/egg/easy_install.py to easy_install.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying distribute.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying distribute.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying distribute.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying distribute.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying distribute.egg-info/entry_points2.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying distribute.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying distribute.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating dist
creating 'dist/distribute-0.6.14-py2.4.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing distribute-0.6.14-py2.4.egg
removing '/usr/lib/python2.4/site-packages/distribute-0.6.14-py2.4.egg' (and everything under it)
creating /usr/lib/python2.4/site-packages/distribute-0.6.14-py2.4.egg
Extracting distribute-0.6.14-py2.4.egg to /usr/lib/python2.4/site-packages
distribute 0.6.14 is already the active version in easy-install.pth
Installing easy_install script to /usr/bin
Installing easy_install-2.4 script to /usr/bin

Installed /usr/lib/python2.4/site-packages/distribute-0.6.14-py2.4.egg
Processing dependencies for distribute==0.6.14
Finished processing dependencies for distribute==0.6.14
After install bootstrap.
/usr/lib/python2.4/site-packages/setuptools-0.6c11-py2.4.egg-info already exists

[root@isvx7 ez_setup-0.9]# which easy_install
/usr/bin/easy_install

Next I used the easy_install tool to install pymongo

[root@isvx7 ez_setup-0.9]# easy_install pymongo
install_dir /usr/lib/python2.4/site-packages/
Searching for pymongo
Reading http://pypi.python.org/simple/pymongo/
Best match: pymongo 2.6.3
Downloading https://pypi.python.org/packages/source/p/pymongo/pymongo-2.6.3.tar.gz#md5=da4a7d6ee47fe30b3978b8805d266167
Processing pymongo-2.6.3.tar.gz
Running pymongo-2.6.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-ouUB4f/pymongo-2.6.3/egg-dist-tmp-5NQcY6
zip_safe flag not set; analyzing archive contents...
Adding pymongo 2.6.3 to easy-install.pth file

Installed /usr/lib/python2.4/site-packages/pymongo-2.6.3-py2.4-linux-x86_64.egg
Processing dependencies for pymongo
Finished processing dependencies for pymongo

Check to see if pymongo can be imported fine.

[root@isvx7 ez_setup-0.9]# python
Python 2.4.3 (#1, Oct 23 2012, 22:02:41)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-54)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymongo
>>>

Tuesday, November 5, 2013

Importing data into a MongoDB database using mongoimport from a csv file

I want to import some data from a xml spreadsheet, and load it into a MongoDB database. To do that, the first step is to convert the xml data into a csv file, and then load that data into a MongoDB database using mongoimport.

Below I tried to import a simple csv file into a mongodb database using mongoimport.

Here is my sample csv file:

[root@isvx7 mongodb_new]# cat projects.csv
ISV Name,Storage Brand,Project,Owner
Oracle,SVC,SVC Chubby nodes and Oracle,Mayur
Oracle,IBM Tape,Oracle backup,Shashank
Oracle,V7000,Oracle 12c on V7000,Mayur
Oracle,DS8870,Easy Tier 5 for Oracle,Mayur

The version of MongoDB that I'm currently running

[root@isvx7 mongodb_new]# mongod -version
db version v2.4.8
Tue Nov 5 16:30:37.367 git version: a350fc38922fbda2cec8d5dd842237b904eafc14

Executing the mongoimport command on the same host running the mongodb server.

[root@isvx7 mongodb_new]# mongoimport --db mydb --collection projects --type csv --file ./projects.csv --headerline
connected to: 127.0.0.1
Tue Nov 5 15:47:08.606 imported 4 objects

As seen on the mongo shell

> show dbs
mydb    0.203125GB
> use mydb
switched to db mydb
> db.projects.find().toArray()
[
        {
                "_id" : ObjectId("5279756a601c0948fdb32f70"),
                "ISV Name" : "Oracle",
                "Storage Brand" : "SVC",
                "Project" : "SVC Chubby nodes and Oracle",
                "Owner" : "Mayur"
        },
        {
                "_id" : ObjectId("5279756c601c0948fdb32f71"),
                "ISV Name" : "Oracle",
                "Storage Brand" : "IBM Tape",
                "Project" : "Oracle backup",
                "Owner" : "Shashank"
        },
        {
                "_id" : ObjectId("5279756c601c0948fdb32f72"),
                "ISV Name" : "Oracle",
                "Storage Brand" : "V7000",
                "Project" : "Oracle 12c on V7000",
                "Owner" : "Mayur"
        },
        {
                "_id" : ObjectId("5279756c601c0948fdb32f73"),
                "ISV Name" : "Oracle",
                "Storage Brand" : "DS8870",
                "Project" : "Easy Tier 5 for Oracle",
                "Owner" : "Mayur"
        }
]
>

Tuesday, October 29, 2013

Oracle 12c: cloning a PDB

Out here I create a clone of testdb1(PDB). The clone testdb2(PDB) is on the same CDB(testdb)

SQL> alter pluggable database testdb1 close;

Pluggable database altered.

SQL> alter pluggable database testdb1 open read only;

Pluggable database altered.

SQL>
SQL> create pluggable database testdb2 from testdb1 FILE_NAME_CONVERT=('/u02/app/oracle/oradata/testdb/testdb1','/u02/app/oracle/oradata/testdb/testdb2');
create pluggable database testdb2 from testdb1 FILE_NAME_CONVERT=('/u02/app/oracle/oradata/testdb/testdb1','/u02/app/oracle/oradata/testdb/testdb2')
*
ERROR at line 1:
ORA-65005: missing or invalid file name pattern for file -
/u02/app/oracle/product/12.1.0/dbhome_1/dbs/soe.dbf

The above error was due to the fact that I had not added the datafile soe.dbf from SOE tablespace into the FILE_NAME_CONVERT list. By adding it as shown below, resolved the issue.

The below operation not allowed from within a pluggable database, and needs to be executed from within the CDB. If we execute it withing testdb1, we will get the following error message: ORA-65040: operation not allowed from within a pluggable database.

SQL> create pluggable database testdb2 from testdb1 FILE_NAME_CONVERT=('/u02/app/oracle/oradata/testdb/testdb1','/u02/app/oracle/oradata/testdb/testdb2','/u02/app/oracle/product/12.1.0/dbhome_1/dbs/soe.dbf','/u02/app/oracle/oradata/testdb/testdb2/soe.dbf');

Pluggable database created.

SQL>

We see below that the datafiles have been added to the directory testdb2

-bash-3.2$ pwd
/u02/app/oracle/oradata/testdb/testdb2
-bash-3.2$ ls
soe.dbf sysaux01.dbf system01.dbf temp01.dbf testdb1_users01.dbf
-bash-3.2$ ls -lrt
total 4397364
-rw-r----- 1 oracle oinstall 20979712 Oct 29 22:50 temp01.dbf
-rw-r----- 1 oracle oinstall 5251072 Oct 29 22:50 testdb1_users01.dbf
-rw-r----- 1 oracle oinstall 272637952 Oct 29 22:50 system01.dbf
-rw-r----- 1 oracle oinstall 650125312 Oct 29 22:50 sysaux01.dbf
-rw-r----- 1 oracle oinstall 3570409472 Oct 29 22:50 soe.dbf
-bash-3.2$

Below we open the newly created clone database testdb2

SQL> alter pluggable database testdb2 open;

Pluggable database altered.

SQL>

Next, we close testdb1 which is in read only node and re-open it again.

SQL> alter pluggable database testdb1 close;

Pluggable database altered.

SQL> SQL> alter pluggable database testdb1 open ;

Pluggable database altered.

SQL>

Below we connect to the CDB (testdb) and see that datafiles for the clone PDB (testdb2) now shows up when listing the datafiles.

-bash-3.2$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.1.0 Production on Tue Oct 29 22:55:05 2013

Copyright (c) 1982, 2013, Oracle. All rights reserved.

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL> show con_name

CON_NAME
------------------------------
CDB$ROOT
SQL> show con_id

CON_ID
------------------------------
1
SQL> select name from v$datafile;

NAME
--------------------------------------------------------------------------------
/u02/app/oracle/oradata/testdb/system01.dbf
/u02/app/oracle/oradata/testdb/sysaux01.dbf
/u02/app/oracle/oradata/testdb/undotbs01.dbf
/u02/app/oracle/oradata/testdb/pdbseed/system01.dbf
/u02/app/oracle/oradata/testdb/users01.dbf
/u02/app/oracle/oradata/testdb/pdbseed/sysaux01.dbf
/u02/app/oracle/oradata/testdb/testdb1/system01.dbf
/u02/app/oracle/oradata/testdb/testdb1/sysaux01.dbf
/u02/app/oracle/oradata/testdb/testdb1/testdb1_users01.dbf
/u02/app/oracle/product/12.1.0/dbhome_1/dbs/soe.dbf
/u02/app/oracle/oradata/testdb/testdb2/system01.dbf

NAME
--------------------------------------------------------------------------------
/u02/app/oracle/oradata/testdb/testdb2/sysaux01.dbf
/u02/app/oracle/oradata/testdb/testdb2/testdb1_users01.dbf
/u02/app/oracle/oradata/testdb/testdb2/soe.dbf

14 rows selected.

SQL>

Oracle 12c: control, data, redo log, and temp files in CDB and PDB

After creating the PDB as shown in the previous post, I created the Order Entry schema using Swingbench.
Below we will see the how the control files, data files, redo log files, and the temp files are used by the CDB (testdb) and the PDB (testdb1)

-bash-3.2$ echo $ORACLE_SID
testdb
-bash-3.2$ sqlplus / as sysdba

SQL*Plus: Release 12.1.0.1.0 Production on Tue Oct 29 09:22:28 2013

Copyright (c) 1982, 2013, Oracle. All rights reserved.

Connected to:
Oracle Database 12c Enterprise Edition Release 12.1.0.1.0 - 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options

SQL>
SQL> show con_name

CON_NAME
------------------------------
CDB$ROOT
SQL> show con_id

CON_ID
------------------------------
1
SQL> select name from v$datafile;

NAME
--------------------------------------------------------------------------------
/u02/app/oracle/oradata/testdb/system01.dbf
/u02/app/oracle/oradata/testdb/sysaux01.dbf
/u02/app/oracle/oradata/testdb/undotbs01.dbf
/u02/app/oracle/oradata/testdb/pdbseed/system01.dbf
/u02/app/oracle/oradata/testdb/users01.dbf
/u02/app/oracle/oradata/testdb/pdbseed/sysaux01.dbf
/u02/app/oracle/oradata/testdb/testdb1/system01.dbf
/u02/app/oracle/oradata/testdb/testdb1/sysaux01.dbf
/u02/app/oracle/oradata/testdb/testdb1/testdb1_users01.dbf
/u02/app/oracle/product/12.1.0/dbhome_1/dbs/soe.dbf

10 rows selected.

SQL> select con_id,name from v$tempfile;

    CON_ID
----------
NAME
--------------------------------------------------------------------------------
         1
/u02/app/oracle/oradata/testdb/temp01.dbf

         2
/u02/app/oracle/oradata/testdb/pdbseed/pdbseed_temp01.dbf

         3
/u02/app/oracle/oradata/testdb/testdb1/temp01.dbf

SQL> select name from v$controlfile;

NAME
--------------------------------------------------------------------------------
/u02/app/oracle/oradata/testdb/control01.ctl
/u02/app/oracle/fast_recovery_area/testdb/control02.ctl

SQL> select member from v$logfile;

MEMBER
--------------------------------------------------------------------------------
/u02/app/oracle/oradata/testdb/redo03.log
/u02/app/oracle/oradata/testdb/redo02.log
/u02/app/oracle/oradata/testdb/redo01.log

SQL>

Next we change the session container to the PDB (testdb1).

SQL> ALTER SESSION SET CONTAINER =testdb1;

Session altered.

SQL> show con_name

CON_NAME
------------------------------
TESTDB1
SQL> show con_id

CON_ID
------------------------------
3
SQL> select name from v$datafile;

NAME
--------------------------------------------------------------------------------
/u02/app/oracle/oradata/testdb/undotbs01.dbf
/u02/app/oracle/oradata/testdb/testdb1/system01.dbf
/u02/app/oracle/oradata/testdb/testdb1/sysaux01.dbf
/u02/app/oracle/oradata/testdb/testdb1/testdb1_users01.dbf
/u02/app/oracle/product/12.1.0/dbhome_1/dbs/soe.dbf

SQL> select con_id,name from v$tempfile;

CON_ID
----------
NAME
--------------------------------------------------------------------------------
3
/u02/app/oracle/oradata/testdb/testdb1/temp01.dbf

SQL> select name from v$controlfile;

NAME
--------------------------------------------------------------------------------
/u02/app/oracle/oradata/testdb/control01.ctl
/u02/app/oracle/fast_recovery_area/testdb/control02.ctl

SQL> select member from v$logfile;

MEMBER
--------------------------------------------------------------------------------
/u02/app/oracle/oradata/testdb/redo03.log
/u02/app/oracle/oradata/testdb/redo02.log
/u02/app/oracle/oradata/testdb/redo01.log

SQL>