Machine Learning Algorithms

⛔ No, PCA is NOT a feature selection procedure. I often get asked why I did not include PCA in my course "Feature Selection for Machine Learning". And the answer is: because it is not a feature selection procedure. Let me explain 👇 ☑️ Feature selection is the process of picking the most relevant features from a dataset. ☑️ The aim of feature selection is to remove redundant features and make models simpler, faster and more interpretable. ☑️The key piece of information: feature selection algorithms do NOT alter the data, they just select the best features from any given dataset. Now, let's talk about Principal Component Analysis (PCA). ✅ PCA is a dimensionality reduction technique used to transform the original features into a new set of orthogonal components, also known as principal components. ✅ PCA is used for compressing data while retaining its variability. The first few components tend to capture most of the variability in the data. ❌ PCA might seem like a feature selection technique, but it's not! ❌ Unlike feature selection, PCA doesn't focus on identifying specific important features. It rearranges the existing ones into a new space based on variance, often leading to loss of interpretability since the components are combinations of original features. ❌ In addition, we still need all original features to create the principal components. So it does not really simplify our deployment pipelines (or select features for that matter). Looking for methods that truly select features? Here are some options: ▶️ Recursive Feature Elimination (RFE): Iteratively removes less important features. ▶️ LASSO: Adds regularization to linear models, effectively shrinking and removing less impactful features. ▶️ Leverage feature importance scores derived from Tree-based methods (e.g., Random Forest) to remove less relevant attributes. ▶️ Permutation feature importance: remove those features that don't affect model performance when randomly shuffled. ▶️ Introduce noise features: Add random features, train a model, and then remove all features whose performance/importance is below the random noise. So to wrap up, the goal of feature selection is to fine-tune your feature set while keeping the essence of your data intact! 📊 To learn more about feature selection, including Python demos, check out my book (https://lnkd.in/dxFJhn-8 ) and my course (https://buff.ly/mRbqF6c). #pca #featureselection #machinelearning #datascience #datascientist #dataengineering #mlmodels #ai

The AI landscape has rapidly evolved beyond just large language models. Today’s systems rely on a wide range of foundational model types—each designed for specific modalities, tasks, and constraints. This visual covers 12 foundational AI models and their core workflows. This is intended for engineers, researchers, and builders who want a structured view of the ecosystem. Here’s a breakdown of what’s included: → LLM (Large Language Models) – GPT, LLaMA Trained using transformer architecture to generate coherent, human-like text. The workflow involves data collection, tokenization, pattern learning, fine-tuning, and deployment. → SLM (Small Language Models) – Phi, TinyLLaMA Lightweight and efficient for on-device or low-resource environments. Focuses on model compression, compact training, and benchmarking. → VLM (Vision-Language Models) – CLIP, Flamingo Learns joint understanding between images and text. Ideal for tasks like image captioning and visual QA. → MLLM (Multimodal Large Language Models) – Gemini Designed to process and align multiple modalities such as text, image, audio, and video. → LAM (Large Action Models) – RT-2, InstructDiffusion Generates sequences of executable actions using behavioral and reinforcement learning data. → LRM (Large Reasoning Models) – DeepSeek-R1 Structured for tool use, chain-of-thought reasoning, and test-time modularity in logic-heavy tasks. → MoE (Mixture of Experts) – Mixtral Activates a subset of specialized models per input to reduce computation cost and improve performance. → SSM (State Space Models) – Mamba, RetNet Efficient at long-context sequence modeling using dynamic systems and parallelism. → RNN (Recurrent Neural Networks) – LSTM, GRU Uses hidden states to process time-dependent data, maintaining memory across input sequences. → CNN (Convolutional Neural Networks) – EfficientNet Learns spatial patterns in image data via convolution layers, pooling, and hierarchical stacking. → SAM (Segment Anything Model) – Meta Segments objects from images based on prompts (text, points, or boxes), making it useful for dynamic image understanding. → LNN (Liquid Neural Networks) – LFMs Leverages differential equations to adapt in real-time, supporting applications in time-sensitive environments. This chart is designed to help you understand not just what these models are, but how they work under the hood. If you're working in AI,  this foundational understanding is crucial for making informed architectural decisions.

Your model is trained. But is it actually good? Most ML engineers default to accuracy. Then wonder why their model fails in production. Here are 20 evaluation metrics — and when to actually use each one: Classification: - Accuracy → Balanced datasets only. - Precision → When false positives are costly. - Recall → When false negatives matter more. - F1 Score → Imbalanced datasets. Balances both. - ROC-AUC → Binary classification evaluation. - Log Loss → Probabilistic models. Penalizes confident wrong predictions. - Confusion Matrix → Error analysis. See exactly where it breaks. - Specificity → When detecting negatives correctly matters. - Balanced Accuracy → Uneven datasets. Don't trust plain accuracy here. Regression: - MAE → Simple, interpretable error measurement. - MSE → Penalizes larger errors more heavily. - RMSE → Error in original scale. Most interpretable. - R² Score → How much variance your model explains. - Adjusted R² → Feature-heavy models. Adjusts for complexity. - MAPE → Business forecasting. Error as a percentage. - Explained Variance → Model consistency evaluation. Clustering: - Silhouette Score → Cluster cohesion and separation. Cluster validation. - Davies-Bouldin Index → Lower is better clustering. NLP: - BLEU Score → Machine translation quality. - ROUGE Score → Text summarization quality. Accuracy is not a strategy. Picking the right metric for the right problem is. A model that looks great on accuracy can destroy real-world outcomes when the wrong metric guided its evaluation. Save this. 📌 Which metric do most engineers misuse? 👇

Most neural networks see the input data as a matrix or a vector, like pixels in rows and columns or words in a sequence. A lot of real-world data is better described as a graph: a set of nodes connected by edges, like social networks, power grids, or molecules. Graph neural networks try to process this kind of data, and the most common approach before this paper (GCNs) treated every neighbor of a node as equally important when aggregating information. This ICLR 2018 paper, published by Bengio's team and since then garnered more than 18,000 citations, introduces Graph Attention Networks (GATs), which borrow the idea of attention in language models to let each node learn which of its neighbors matter more for a given task. The attention weights are computed using a small shared neural network that looks at pairs of connected nodes, so the model doesn't need to know the full graph structure ahead of time and can generalize to entirely new graphs it's never seen during training. It's an exemplary AI research paper. The architecture is simple enough to describe in a few equations, runs efficiently in parallel, and at the time matched or beat everything else on standard benchmarks. Read online with an AI tutor: https://lnkd.in/e2a_RBht Read offline on your own: https://lnkd.in/eFVrT2kz

🚨 Singapore published a case study applying its Agentic AI Framework (the world's first of its kind) to OpenClaw. [Download it below]. If you use AI agents, check out these SAFETY best practices: 1. Assess and bound the risks upfront - Avoid deploying OpenClaw in its open-source form in mission-critical environments - Avoid creating a single “all-powerful” OpenClaw agent with unrestricted access - Avoid installing OpenClaw on primary work or personal devices that contain sensitive data - Avoid granting OpenClaw ‘superuser’ privileges - Avoid granting OpenClaw unrestricted access to files and applications 2. Make humans meaningfully accountable - Adopt a risk-based approach to determine the appropriate level of agent autonomy with the sensitivity of data and the criticality of tasks - Identify checkpoints that require human approval - Enforce human approval through system-level controls where possible 3. Implement technical controls and processes a) During design and development - Enforce control-plane separation for key safety controls - Route outbound connections through a policy-enforcing proxy - Review and tighten the OpenClaw configurations, which are permissive by default - Avoid giving OpenClaw access to sensitive data - Use dedicated identities and credentials for the agent - Avoid exposing credentials to OpenClaw directly - Regularly rotate API keys, OAuth tokens, and other credentials used by the agent - Use trusted skills only - Use trusted sources b) Testing before deployment - Adopt a structured evaluation approach, organized around capability-based risk identification, concrete risk scenarios, as well as environment and tool mapping - Test and verify that safety controls are working as intended - Test and verify that human-in-the-loop (HITL) is working as intended - Test and verify that safeguards remain effective against indirect prompt injections, especially when third-party skills are used c) Post deployment - Ensure that all agent actions are logged and attributable - Avoid leaving the agent unsupervised for extended periods - Monitor the agent for behavioral anomalies and policy violations - Treat rebuild as an expected control, especially in the event of compromise or anomalous behavior - Regularly update OpenClaw and patch known vulnerabilities promptly 4.  Enable end-user responsibility - Provide personnel training and/or clear usage guidance - 👉 This is a super interesting case study, and a must-read for those developing or deploying AI agents. Download it below. 👉 To learn more and stay up to date, join my newsletter's 95,200+ subscribers (link below).

“99% Accuracy” from an ML model is a lie. 🚨 If you are building a fraud detection model and 99% of your transactions are legitimate, a model that simply guesses "Legit" every single time will have 99% accuracy. But it captures 0% of the fraud. In the real world, "Accuracy" is rarely the best metric. If you want to move from a junior developer to a Senior ML Engineer, you need to understand the nuances of how we measure success. In this visual story lets get on ML evaluation journey. 🛑 Stop 1: Classification (The "Is it X or Y?" problems) • Precision: When you predict "Spam," how often are you right? (Crucial when false alarms are annoying). • Recall: Out of all the actual "Spam," how much did you find? (Crucial when missing a positive is dangerous). • F1 Score: The harmonic mean. It’s the peace treaty between Precision and Recall. 🛑 Stop 2: Regression (The "How much?" problems) • MAE (Mean Absolute Error): The average "oops." Great for generic error tracking (e.g., House prices off by $5k). • MSE (Mean Squared Error): This penalizes large errors heavily. Use this if being very wrong is much worse than being slightly wrong. • RMSE: Puts the error back into the same units as the target so you can actually explain it to your boss. 🛑 Stop 3: Clustering & Ranking • Silhouette Score: Are your customer segments actually distinct, or just a messy blob? • ROC-AUC: How well does the model separate classes? (e.g., Distinguishing Fraud vs. Not Fraud). Don't just optimize for the high score. Optimize for the business problem. Save this roadmap for your next model deployment! 💾 Like this? Share it and follow me Priyanka for more cloud and AI concepts. #MachineLearning #DataScience #AI #DeepLearning

Most people building with AI today have never really thought about GANs. But if you want to understand how modern AI systems learn to get better, GANs are one of the most important ideas to know. A Generative Adversarial Network (GAN) works by training two models together: 1/ Generator: creates synthetic data from noise 2/ Discriminator: tries to detect whether the data is real or fake The generator improves by trying to fool the discriminator. The discriminator improves by learning to catch the generator’s mistakes. This competitive loop pushes both models to improve. What many people miss is that GANs were never just about generating images. But long before generative AI became mainstream, GANs were already being used in practical machine learning systems. When I was working as a data scientist at IBM, we experimented with GAN-style setups to improve model performance. In practice, this meant the system was constantly being exposed to harder edge cases instead of only clean training data, which helped improve the model’s accuracy and generalization. The generator creates harder problems, and the model becomes smarter by learning to solve them. That’s why understanding GANs still matters today. You’ll see these ideas show up in: ✦ Synthetic data generation ✦ Data augmentation for small datasets ✦ Image super-resolution ✦ Medical imaging and drug discovery ✦ Robustness testing for ML systems Even though diffusion models and LLMs dominate the conversation today, the core idea of adversarial learning is still everywhere in modern AI systems. So if you find yourself at a cross-roads where the model accuracy is stunted, try using GAN

Kolmogorov-Arnold Networks as an alternative to traditional Neural Networks! Researchers from MIT, Caltech, and Northeastern have introduced a new type of neural network architecture known as Kolmogorov-Arnold Networks (KANs), which presents a significant challenge to the traditional use of Multi-Layer Perceptrons (MLPs). KANs offer a novel approach to neural network architecture inspired by the Kolmogorov-Arnold representation theorem. This theorem essentially states that any multivariate continuous function can be represented as a composition of univariate functions and the addition operation. Translating this into neural network design, KANs uniquely place adaptable activation functions on the connections or edges between nodes rather than using standard fixed activation functions at the nodes themselves. This flexibility allows KANs to potentially model complex relationships and patterns more effectively, as they can tailor the transformation at each connection to better suit the specific data and task at hand, diverging from traditional networks where the choice of activation function at each layer is static and uniform across the network. In terms of accuracy, much smaller KANs can achieve comparable or better performance than larger MLPs on tasks such as data fitting and PDE solving. Moreover, KANs demonstrate faster neural scaling laws, meaning their performance improves more rapidly with increased model size compared to MLPs. KANs also excel in interpretability. They can be intuitively visualized and allow for easy interaction with human users. In case studies from knot theory and physics, KANs served as interactive "collaborators" to help scientists rediscover known mathematical and physical laws, showcasing their potential for scientific discovery. KANs could potentially serve as a foundation model for AI+Science applications and open opportunities to improve today's deep learning models that heavily rely on MLPs. Read the full paper for more details: https://lnkd.in/erEF6HbT :)

If you’ve ever shipped a GenAI model to production, you already know the real interview isn’t about transformers, it’s about everything that breaks the moment real users touch your system. 1) How would you evaluate an LLM powering a Q&A system? Approach: Don’t talk about accuracy alone. Break it down into: ✅ Functional metrics: exact match, F1, BLEU, ROUGE depending on task. ✅ Safety metrics: hallucination rate, refusal rate, PII leakage. ✅ User-facing metrics: latency, token cost, answer completeness. ✅ Human evaluation: rubric-based scoring from SMEs when answers aren’t deterministic. ✅ A/B tests: compare model variants on real user flows. 2) How do you handle hallucinations in production? Approach: ✅ Show you understand layered mitigation: ✅ Retrieval first (RAG) to ground the model. ✅ Constrain the prompt: citations, “answer only from provided context,” JSON schemas. ✅ Post-generation validation like fact-checking rules or context-overlap checks. ✅ Fall-back behaviors when confidence is low: ask for clarification, return source snippets, route to human. 3) You’re asked to improve retrieval quality in a RAG pipeline. What do you check first? Approach: Walk through a debugging flow: ✅ Check document chunking (size, overlap, boundaries). ✅ Evaluate embedding model suitability for domain. ✅ Inspect vector store configuration (HNSW params, top_k). ✅ Run retrieval diagnostics: is the top_k relevant to the question? ✅ Add metadata filters or rerankers (cross-encoder, ColBERT-style scoring). 4) How do you monitor a GenAI system after deployment? Approach: ✅ Make it clear that monitoring isn’t optional. ✅ Latency and cost per request. ✅ Token distribution shifts (prompt bloat). ✅ Hallucination drift from user conversations. ✅ Guardrail violations and safety triggers. ✅ Retrieval hit rate and query types. ✅ Feedback loops from thumbs up/down or human review. 5) How do you decide between fine-tuning and using RAG? Approach: ✅ Use a decision tree mentality: ✅ If the issue is knowledge freshness, go with RAG. ✅ If the issue is formatting/style, go with fine-tuning. ✅ If the model needs domain reasoning, consider fine-tuning or LoRA. ✅ If the data is large and structured, use RAG + reranking before touching training. Most interviews test what you know. GenAI interviews test what you’ve survived. Follow Sneha Vijaykumar for more... 😊 #genai #datascience #rag #production #interview #questions #careergrowth #prep

Spatial data is a goldmine, but how you engineer and select those features can make or break your machine learning model. Here is my F to S Tier list of using spatial features in Data Science: F-Tier: Raw Lat/Lon into Tree-Based Models - Feeding raw Latitude and Longitude directly into an XGBoost or Random Forest model. Trees make orthogonal splits, meaning they struggle to isolate specific locations. Not much comes of this. D-Tier: One-Hot Encoding ZIP Codes - Turning 10,000 ZIP codes or neighborhoods into dummy variables. Zip code numbers tell you nothing about the areas they represent (and are postal routes not actual boundaries) and you have completely erased the actual physical distance between those locations. C-Tier: Distance to Points of Interest (POIs) Calculating the Haversine or Manhattan distance to key landmarks (city centers, transit stops, competitors). This is a better, a reliable baseline that actually gives your model actionable context about where something is relative to what matters. B-Tier: Spatial Indexing (Geohash / H3) - Converting coordinates into localized bins like A5 or Uber’s H3 Hexagons. This allows you to aggregate historical target variables by region and captures non-linear spatial relationships without blowing up the feature space. A-Tier: Spatial Lags & Graph Embeddings - Encoding the "neighborhood effect." Instead of just looking at the point itself, you calculate Spatial Lags (e.g., the average target value of the 5 nearest neighbors) or use Graph Neural Networks (GNNs) to create spatial embeddings. This explicitly models the rule that things close together are more related than things far apart. AKA use PySAL. S-Tier: Spatial Regression for Feature Selection - Standard feature importance metrics (like SHAP or tree-based Gini) will lie to you if spatial autocorrelation is present. Using spatial regression technique like Geographically Weighted Regression (GWR) or Spatial Error Models (SEM) to filter your features ensures you aren't just keeping redundant variables that act as secret proxies for location. It is the ultimate reality check for your spatial dataset. But let me know what you do with your data - would love to hear other ideas. 🌎 I'm Matt Forrest and I talk about modern GIS, earth observation, AI, and how geospatial is changing. 📬 Want more like this? Join 12k+ others learning from my daily newsletter → forrest.nyc