E-Learn Knowledge Base
Dot Product
The Dot Product, or the inner product or scalar product, is a fundamental operation in vector algebra that combines two vectors to produce a scalar quantity. This operation is essential in various fields, including physics, engineering, and computer graphics, due to its ability to measure the similarity between two vectors. The dot product can provide information about the angle between vectors, the projection of one vector onto another, and more.
1.0Dot and Cross Product of a Vector
What is the Dot Product?
The Dot Product ⋅b is calculated as the sum of the products of corresponding components of two vectors.
The dot product is also known as the inner product or the scalar product and reads “a dot b”.
If =x1i^+y1j^+z1k^ and =x2i^+y2j^+z1k^ then dot product is ⋅b=x1x2+y1y1+z1z2.
The scalar product is useful for determining the angle between two vectors.
Geometric Interpretation
Geometrically, the dot product can be interpreted as:
⋅B=∣A∥B∣cos(θ)
Where are the magnitudes of the vectors, and is the angle between them.
What is the Cross Product?
Given two vectors =(A1, A2, A3) and =(B1, B2, B3), the cross product ×B is defined as:
×B=(A2 B3−A3 B2, A3 B1−A1 B3, A1 B2−A2 B1)
This resultant vector is orthogonal to both .
Geometric Interpretation
The magnitude of the cross product is given by: ×B∣=∣A∥B∣sin(θ) where and are the magnitudes of vectors and , and θ is the angle between them determines the direction of the resulting vector, following the right-hand rule.
2.0Dot Product Formula
For two vectors =(A1, A2, A3) and =(B1, B2, B3) in three-dimensional space, the dot product mathbf{A} cdot mathbf{B}is defined as:
⋅B=A1 B1+A2 B2+A3 B3
3.0Dot Product Rules
The dot product, commonly known as the scalar product, is a foundational operation in vector algebra with several important properties and rules. Here are the key rules and properties of the dot product:
-
Commutative Property
The dot product of 2 vectors is commutative, meaning the order in which you take the dot product does not matter:
⋅B=B⋅A
-
Distributive Property
The dot product distributes over vector addition:
⋅(B+C)=A⋅B+A⋅C
-
Scalar Multiplication
If you multiply a vector by a scalar and then take the dot product, it is equivalent to taking the dot product first and then multiplying by the scalar:
kA)⋅B=k(A⋅B)A⋅(kB)=k(A⋅B)
-
Zero Vector
The dot product of any vector with the zero vector results in zero:
⋅0=0
-
Orthogonal Vectors
If two vectors are orthogonal (perpendicular), their dot product is zero:
⋅B=0 if A⊥B
-
Magnitude Relation
The dot product of a vector with itself gives the square of its magnitude: ⋅A=∣A∣2 where is the magnitude of .
-
Angle Between Vectors
The dot product can be used to find the cosine of the angle θ between two vectors:
⋅B=∣A∣∣B∣cos(θ) This relation can be rearranged to find the angle: (θ)=∣A∥B∣A⋅B
4.0Dot Product of a Vector with Itself
For a vector =(A1, A2, A3) in three-dimensional space, the dot product of the vector with itself is given by: ⋅A=A12+A22+A32
5.0Dot Product of Parallel Vectors
When the two vectors are parallel then
- Same Direction:
When vectors are parallel and in the same direction, =0∘. Therefore, (0∘)=1:A⋅B=∣A∥B∣
- Opposite Direction:
When vectors A and B are parallel but in opposite directions, θ=180. Therefore,
(180∘)=−1:A⋅B=−∣A∥B∣
6.0Dot Product of Perpendicular Vectors
If 2 vectors are Perpendicular, then their dot product is zero:
⋅B=0
Geometric Interpretation
Perpendicular vectors form a 90∘ angle between them. In terms of the dot product, this means:
(90∘)=0
Thus, ⋅B=∣A∥B∣cos(90∘)=0.
7.0Dot Product of Unit Vectors
A unit vector ^ is a vector with a magnitude of 1. For example, in three-dimensional space, the unit vectors are ^=(1,0,0),j^=(0,1,0),andk^=(0,0,1).
- Same Unit Vector
The dot product of a unit vector with itself is always 1: ^⋅u^=1
This is because the magnitude of a unit vector is 1. i.e. u^∣=1
- Different Unit Vectors
The dot product of 2 different unit vectors depends on their orientation relative to each other: ^⋅v^=cos(θ), where θ is the angle between the two-unit vectors.
- If ^ and ^ are perpendicular θ=90∘), then: ^⋅v^=0.
- If ^ and ^ are parallel θ=0∘ or 180∘), then: ^⋅v^=1
8.0Dot Product Example
Example 1: Find the dot product of 2 vectors =[3,−2,5],B=[4,0,−1]
Solution:
⋅B=AxBx+AyBy+AzBz
⋅B=(3)(4)+(−2)(0)+(5)(−1)
⋅B=12+0−5=7
So, the dot product of A × B is 7.
Example 2: Let there be two vectors |a| = 10 and |b| = 5 and θ = 45°. Find their dot product.
Solution: As we know the formula of dot product
a × b = |a| |b| cos θ
a × b = (10) (5) cos 45°
50×21=252
Example 3: Given vector A = [1, 2, 3] and B = [4, –5, 6], Find the dot product of these two vectors.
Solution: As we know the formula of dot product
a × b = |a| |b| cos θ
a × b = (1) (4) + (2) (–5) + (3) (6)
= 4 – 10 + 18 = 12
A∣=1+4+9=14
B∣=16+25+36=77
cosθ=∣a∣∣b∣a⋅b=147712≈32.8212≈0.3656
θ = cos–1(0.3656) » 68.56°.
Example 4: Calculate the dot product of the two vectors and angle between them A=(-1,2,-2) quad B=(6,3,-6)
Solution: Dot product
⋅B=AxBx+AyBy+AzBz
⋅B=(−1)(6)+(2)(3)+(−2)(−6)
= –6 + 6 + 12 = 12
By dot product formula
⋅B=∣A∣∣B∣cosθ
A∣=1+4+4=9=3
B∣=36+9+36=81=9
cosθ=∣A∣∣A∣A⋅B
θ=3×912=94
=cos−1(94).
Example 5: Calculate the dot product of vectors =2i^+5j^−k^ and =i^−j^−3k^ and the angle between them.
Solution: ⋅B=AxBx+AyBy+AzBz
A × B = (2) (1) + (5) (–1) + (–1) (–3)
= 2 – 5 + 3 = 0
Dot product formula
⋅B=∣A∣∣B∣cosθ
A∣=4+25+1=30
B∣=1+1+9=11
θ=∣A∣∣B∣A⋅B=30110
os θ = 0
θ = 90°
So, the dot product of two vectors A & B is 0 and the angle between them is 90°.
9.0Dot Product Practice Questions
Find the dot product of the following vectors and determine the angle between them.
- A = (2, 4, 1) and B = (3, 5, 7)
- A = (–2, 3, 11) and B = (5, 7, –4)
- =2i^+5j^−k^andB=i^−2j^+k^
10.0Solved Questions on Dot Product
Q. What is the dot product?
Ans: The dot product is also known as the inner product or the scalar product and reads “a dot b”.
If =x1i^+y1j^+z1k^ and =x2i^+y2j^+z1k^ then dot product is ⋅b=x1x2+y1y1+z1z2.
Q. How do you calculate the dot product of 2 vectors?
Ans: To calculate the dot product of two vectors, multiply the corresponding components of the vectors and then sum the products. For vectors =(A1, A2, A3) and =(B1, B2, B3):
⋅B=A1 B1+A2 B2+A3 B3
Q. How is the dot product geometrically interpreted?
Ans: Geometrically, the dot product of two vectors and can be interpreted as: ⋅B=∣A∥B∣cos(θ) where are the magnitudes of the vectors, and is the angle between them.
Q. What does it mean if the dot product is zero?
Ans: If the dot product of 2 vectors is zero, it means that the vectors are orthogonal (perpendicular) to each other. This is because the cosine of is zero: ⋅B=0⇒θ=90∘
Q. What are the properties of the dot product?
Ans: The dot product has several important properties:
Commutative: ⋅B=B⋅A
Distributive: ⋅(B+C)=A⋅B+A⋅C
Scalar Multiplication: cA)⋅B=c(A⋅B)
Q. How does the dot product relate to the projection of one vector onto another?
Ans: The projection of vector onto vector is given by:
BA=(B⋅BA⋅B)B
The dot product ⋅B is crucial in determining the magnitude of this projection.
Authors: T. C. OkennaRegister for this course: Enrol Now
Register for this course: Enrol Now
Register for this course: Enrol Now
Data Analysis with Python
There are many programming languages available, but Python is popularly used by statisticians, engineers, and scientists to perform data analytics.
Here are some of the reasons why Data Analytics using Python has become popular:
- Python is easy to learn and understand and has a simple syntax.
- The programming language is scalable and flexible.
- It has a vast collection of libraries for numerical computation and data manipulation.
- Python provides libraries for graphics and data visualization to build plots.
- It has broad community support to help solve many kinds of queries.
Python Libraries for Data Analytics
One of the main reasons why Data Analytics using Python has become the most preferred and popular mode of data analysis is that it provides a range of libraries.
NumPy: NumPy supports n-dimensional arrays and provides numerical computing tools. It is useful for Linear algebra and Fourier transform.
Pandas: Pandas provides functions to handle missing data, perform mathematical operations, and manipulate the data.
Matplotlib: Matplotlib library is commonly used for plotting data points and creating interactive visualizations of the data.
SciPy: SciPy library is used for scientific computing. It contains modules for optimization, linear algebra, integration, interpolation, special functions, signal and image processing.
Scikit-Learn: Scikit-Learn library has features that allow you to build regression, classification, and clustering models.
Now, let’s look at how to perform data analytics using Python and its libraries.
Data Analytics Using the Python Library, NumPy
Let’s see how you can perform numerical analysis and data manipulation using the NumPy library.
1. Create a NumPy array.
2. Access and manipulate elements in the array.
3. Create a 2-dimensional array and check the shape of the array.
4. Access elements from the 2D array using index positions.
5. Create an array of type string.
6. Using the arange() and linspace() function to evenly space values in a specified interval.
7. Create an array of random values between 0 and 1 in a given shape.
8. Create an array of constant values in a given shape.
9. Repeat each element of an array by a specified number of times using repeat() and tile() functions.
10. Create an identity matrix using eye() and identity() function.
11. Create a 5x5 2D array for random numbers between 0 and 1.
12. Sum an array along the column.
13. Sum an array along the row.
14. Calculate the mean, median, standard deviation, and variance.
15. Sort an array along the row using the sort() function.
16. Append elements to an array using the append() function.
17. Delete multiple elements in an array.
18. Concatenate elements from 2 arrays.
Data Analytics Using Python Libraries, Pandas and Matplotlib
We’ll use a car.csv dataset and perform exploratory data analysis using Pandas and Matplotlib library functions to manipulate and visualize the data and find insights.
1. Import the libraries.
2. Load the dataset using pandas read_csv() function.
3. Display the head of the dataset using the head() function.
4. Display the bottom 5 rows from the dataset using the tail() function.
5. Print summary statistics of the dataset using the describe() function.
6.Plot a histogram for all the variables.
7. Box plot to visualize the relationship between vehicle size and engine hp.
8. Build a pair plot using the seaborn library.
9. Drop irrelevant columns from the dataset using drop() function.
10. Use rename() function to rename the columns.
11. Print the total number of duplicate rows.
12. Remove the duplicate rows using the drop_duplicates() function.
13. Drop the missing values from the dataset.
14. Plot a histogram to find the number of cars per brand.
15. Draw a correlation plot between the variables.
Conclusion
Data is getting generated rapidly in various formats. And companies are relying on data analytics to derive valuable information and hidden insights from this data. After reading this ‘Data analytics using Python’ article, you would have learned what data analytics is and the various applications of data analytics. You also looked at the different types of data analytics and process steps. Finally, you performed data analytics using Python’s NumPy, Pandas, and Matplotlib libraries.
Authors: Avijeet Biswal, T. C. OkennaRegister for this course: Enrol Now
Vectors for Data Science in Python
With the democratization of AI/ML and open source libraries like Keras, scikit-learn etc, anyone with basic python knowledge can set up a working ML classifier in under 5 mins time. While this is more than enough to get started, if you want to understand how different ML algorithms work or implement the latest SOTA (State of the Art) papers to your particular domain, the lack of mathematical expertise quickly becomes a bottleneck as I have experienced firsthand.
In this set of articles, I would try to introduce fundamental mathematics concepts one at a time for non math audience and show it’s practical use in ML / AI domain.
We start off with the simplest of the lot, vectors.
Vectors are simply quantities with direction. A few relatable real world example of vectors are Force, velocity, displacement etc.
For moving a shopping cart, you need to push (apply force) in the direction you want to move the cart. The force expended by you in moving the cart can be described fully by two values, the intensity (magnitude) of the push and the direction you pushed the cart. Any such quantity which requires both magnitude and direction to describe completely is called a vector.
Vectors are usually represented as bold lower case characters like v, w etc. **** Since writing boldface characters using pen and paper is difficult, it’s also represented with an arrow on top of lower case characters when using pen and paper. For this article, we will stick with boldface representation.
Graphically vectors are represented as arrows whose length signify the magnitude (intensity) of the vector and whose angle (from a frame of reference; in this case horizontal )represent the direction of the vector as shown below.
Please note that it is not required that the vector should start from origin (0,0). They can start from any point. For e.g. In the above diagram, u = w and v = a since they have same magnitude and same direction.
There are many ways to represent vectors mathematically. As Data scientists, the one we are interested in is to represent them as a tuple of numbers. Thus vector u can be represented as (2,2) while vector v can be represented as (4, 1). Same holds true for vector w and a.
Though it is easy for us to visualize vectors in 2 and 3 dimensions, the concept of vectors is not limited to 2 and 3 dimensions. It can be generalized to any number of dimensions and this is what makes vectors so useful in Machine Learning.
For e.g. c = (2,1,0) represent a vector in 3 dimensional space while d = (2,1,3,4) represent a vector in 4 dimensional space. As humans, though we cannot visualize dimensions higher than 3, mathematical way of representing vectors gives us the ability to perform operations on higher dimensional vector space.
By now, you must be bored and wondering why as a ML enthusiast you need to learn elementary physics and vectors. Turns out vectors have multiple applications in machine learning from building recommendation engines, to numerical representation of words for Natural Language processing etc and forms the base for all Deep Learning models for NLP.
Let’s start with a code example of how vectors are implemented in numpy and tensorlfow.
Please note: Full code is made available as gist on last section. Relevant subsections are inserted as pictures for illustration purpose.
Below you can see a simple word2vec implementation to show practical use of vectors in Natural Language Processing
As you can see, storing words in higher dimensional vector format is one of the main applications of vectors in Natural Language processing. This type of embedding preserves the context of the word.
In next section, we will go through basic operations like addition and subtraction and how it applies on vectors.
Vector Addition
Now that we have defined what a vector is, let’s find out how to perform basic arithmetic operations on them.
Let’s take the same two vectors u and v and perform a vector addition on them.
To add two vectors u and v graphically, we move the vector v such that it’s tail starts at the head of vector u as shown above (Lines DE and EF ). The sum of two vectors is the vector b that starts at tail of u and ends at the head of v (Line DF).
For better intuition, let’s take a real world example of driving to grocery shop. On the way, you stopped at gas station to fill gas. Let’s assume vector u represents how far the gas station (Point E ) is from your home (point D). If vector u represent the distance (displacement) from gas station to grocery store, then vector b, drawn from the tail of u to the head of v represent the sum u + v. It **** represents how far the grocery store (Point F) is from your home (initial starting point D).
Mathematically b can be represented as (6, 3) by looking at the graph.
There are other methods of calculating vector addition graphically like parallelogram method which you can explore on your own.
Now it’s not possible to plot a graph every-time we want to do vector arithmetic especially when it comes to higher dimension vectors. Fortunately, the mathematical representation of vectors provide us an easy way of doing vector addition.
Since each vector is a tuple of numbers, let’s see what we get if we add the corresponding numbers of each vector.
In the example above, u = (2,2) v = (4,1) b = (2+4, 2+1) = (6,3) which is same as the solution obtained graphically.
Thus vector addition can be done by simply adding corresponding elements of each vector and as you might have already inferred, only vectors having same dimensions can be added together. Let’s see a code implementation
Vector Subtraction
Before we move on to vector subtraction, let’s take a quick look at scalar multiplication another useful property of vectors.
Scalar is nothing but a quantity with only magnitude and no direction. e.g. any integer is a scalar. A real world example of scalar quantity is mass (weight), height of a person etc.
Let’s see what happens when we multiply a vector with a scalar quantity.
u = (2,2) v = (4,1)
If we want to multiply u with a scalar quantity C = 3, one intuitive way to look at it would be to multiply the individual numbers within vector u (2,2) with 3. Let’s see how that looks
d = C x u = 3 x u = (3 x 2, 3 x 2) = (6, 6)
Let’s plot the vectors on a graph and see.
As you can see, multiplying a vector u with a positive scalar value results in a new vector d in same direction, but with magnitude scaled by a factor C = 3
Let’s try multiplying a vector with negative value C = -1
e = C x v = -1 x v = (-1 x 4, -1 x 1) = (-4, -1).
Let’s plot and see how that looks like.
As you can see, multiplying a vector v by -1 results in a vector with same magnitude, but in opposite direction which can be represented as
e = -v or e + v = 0 (Null vector)
Vector subtraction graphically can be considered as a special case of vector addition where u -v = u + -v
Solving graphically
w = u a = -v b = w + a = u + -v = u -v = (-2, 1)
As is evident from graph, b = c or vector subtraction u -v is equal to vector c drawn from head of v to head of u (distance between heads)
Intuitively, this makes sense and is also consistent with traditional number system.
7 -5 = 2 (where 2 is the quantity when added to 5 gives 7) 5 + 2 = 7
Similarly, if you look at the graph, c is the vector which when added to v gives the vector v u = v + c
Now lets do do this mathematically by subtracting individual components within two vectors. c = u -v = (2,2) -(4,1) = (2 -4, 2–1) = (-2,1)
The result is same as the graphical method of solving. A code example is given below for reference.
Register for this course: Enrol Now