Goal:

Be sure that a measured effect actually exists. Quantify how confident we are that this effect exists.

Interpretation

A p-value is the percent chance to see these sort of values if there is no effect, and the new values actually have the same underlying distribution. A p value of 0.05 means that there is a 5% chance that the effect measured doesn't exist.

The hypothesis, that an effect doesn't exists is called the null hypothesis

p values are between 0 and 1. The closer they are to 0, the more certain we are, that the effect exists. A common threshold is 0.05, where we assume that it is certain, that the effect exists.

Example

I am not writing an exact formula here. It depends on your data, but its basically a bayesian formula: p = P(data|there is no relationship between features).

Coin toss example

P value for coin tosses:

If I see 100 coin tosses and I see 60 heads, 40 tails.
P(>60 heads|The coins are independent): my p value

Drug trial example

You test a drug. You have a placebo group and an actual group. You then get the results from both groups. If the results from the actual testing group are better than the placebo group, the question you should be asking is:
"If the drug were ineffective, what is the chance of observing the current results?"

If that chance is above 5% the results are usually considered to be statistically meaningful enough.

Implementation

this implementation assumes normal distribution for both group1 and group2

import numpy as np
from scipy import stats

# Generate some random data for two groups
group1 = np.random.normal(size=30)
group2 = np.random.normal(size=30)

# Perform an independent two-sample t-test
t_stat, p_value = stats.ttest_ind(group1, group2)
print("p-value:", p_value)