Solve the curse of dimensionality by implementing the PCA algorithm on a high-dimensional

-

import numpy as np
import pandas as pd
from sklearn.datasets import make_classification
from sklearn.decomposition import PCA
import matplotlib.pyplot as plt


# Step 1: Generate a High-Dimensional Dataset
# Create a synthetic dataset with 100 features
X, y = make_classification(n_samples=500, n_features=100, n_informative=10, n_redundant=20, random_state=42)
# Convert the data to a DataFrame for easy manipulation
data = pd.DataFrame(X)
print("Original Data Shape:", data.shape)


# Step 2: Apply PCA for Dimensionality Reduction
# Specify the number of components to retain (e.g., keep 2 components for visualization)
pca = PCA(n_components=2)
reduced_data = pca.fit_transform(data)
# Check the shape of the reduced data
print("Reduced Data Shape:", reduced_data.shape)


# Step 3: Check Explained Variance
# This shows how much variance is retained by the selected components
explained_variance = pca.explained_variance_ratio_
print("\nExplained Variance by each principal component:", explained_variance)
print("Total Explained Variance:", np.sum(explained_variance))


# Step 4: Visualize the Reduced Data
# Scatter plot of the two principal components
plt.figure(figsize=(8, 6))
plt.scatter(reduced_data[:, 0], reduced_data[:, 1], c=y, cmap='viridis', marker='o', label='Data Points')
plt.xlabel("Principal Component 1")
plt.ylabel("Principal Component 2")
plt.title("PCA on High-Dimensional Data")
plt.colorbar(label='Class Label')
plt.legend()
plt.show()


Comments

Post a Comment

Popular posts from this blog

Keras

-
  Understanding Keras Models: Your Neural Network Blueprint Think of a Keras model as the blueprint or architecture for building neural networks. Keras is designed to be simple and lets you create models quickly, while running on powerful engines like TensorFlow in the background. The Three Ways to Build a Keras Model 1. Sequential API: The Straightforward Stack What it is: The easiest way to build a model, like stacking Lego blocks in a single, straight line. Best for: Standard networks where data flows directly from one layer to the next with no fancy detours. Limitation: It's too rigid for complex designs that need multiple inputs, outputs, or layers that share connections. 2. Functional API: The Flexible Designer What it is: A more powerful approach that lets you create complex, branch-like architectures. Best for: Advanced designs with multiple inputs/outputs, skip connections (like shortcuts in the network), or layers that are shared. Example: Building sophisticated models li...
Read more

About me

-
Image
  Associate Professor Anil Neerukonda Institute of Technology & Sciences Sangivalsa, Visakhapatnam, A.P., INDIA. VIDWAN | ORCID | SCOPUS | RESEARCH ID | GOOGLE ID
Read more

Principal Component Analysis

-
  Principal Component Analysis (PCA) is a dimensionality reduction technique that transforms high-dimensional data into a lower-dimensional form while preserving as much of the original data’s variance as possible. PCA achieves this by creating new, uncorrelated variables called principal components , which are linear combinations of the original variables. These principal components capture the directions of maximum variance in the data, with the first few components typically containing most of the information. How PCA Works Standardize the Data : Center and scale the data so that each feature has a mean of zero and a variance of one. This step ensures that features with larger scales don’t dominate the results. Compute the Covariance Matrix : Calculate the covariance matrix to understand how features vary with respect to each other. Calculate Eigenvalues and Eigenvectors : Determine the eigenvalues and eigenvectors of the covariance matrix. Eigenvectors define the directions (...
Read more