Database Principles - Keeping Things Normal

Lately, I keep busy by acting as web administrator to a website. Its important however to keep your data categorized. Here are some tips on how to do so...

Atomicity
Consistency
Isolation
Durability

These are the basics, but I don't have time to elaborate now crap dammit (to be continued)

MVC and the Web

Here are some brief notes on MVC and the Web taken from "Design Patterns" by O'Reilly publishing.

1) You make an HTTP request, which is received by a servlet

Using your web browser, you make an HTTP request (usually with some data, like a user name and password). A servlet receives this form of data and parses it.

2) The Servlet acts as the controller

The servlet plays the role of the controller and processes your requests, most likely making requests on the model (usually a database). The result of processing the request is usually bundled up in the form of a JavaBean.

3) The controller forwards control to the view

The View is represented by a JSP. The JSP's only job is to generate the page representing the view of the model (via a Javabean) along with any controls needed for further actions.

4) The view returns a page to the browser via HTTP

A page is returned to the browser, where it is displayed as the view. The user submits further requests, which are processed in the same fashion.

A Java Server Page:

•Enables development of a dynamic user interface

•Is an extension of a Servlet

•Is a combination of:

–Snippets of Java code

–Hyper Text Markup Language (HTML)

–JavaScript

–Cascading Style Sheets (CSS)

--What is a Design Pattern? If not a solution to a problem in a context? (O'Reilly pg. 579)

Model View Controller

Model

•Knows about the data that needs to be displayed.

•Knows about the operations that can be applied on the data.

•Does not know how to display the data to the user. 

View

Really just the observer pattern.

•Provides Graphical User Interface components

•Relays user requests

•Uses the query methods

•Displays information

•Maintains consistency

Controller

•Translates interactions with the view into actions.

•Is responsible for mapping end-user action to application response.

Hashing

Hashing is all about adding integrity to information.

The Hash takes a fixed amount of data and maps it to a new piece of data of fixed sized.

It doesn't matter how big the source data is, the hash will be the same size..

Hashes are NOT encrypted versions of the original texts.

Properties:

1. Hashes are one way. You can't reverse the hash.
2. They are deterministic.
1. One letter change leads to a completely different result. 

Common Uses:

1. Verify a download file
1. Compare the downloaded file hash with the posted hash value. So if the file is corrupt (because a SINGLE byte of data is missing), we know something is wrong.
2. Passwords
1. Since two different messages have a completely different output, even if a single character changes, they are great for passwords. Why store the password when you can store the hashes and compare, that way people who might see the hash CANNOT guess your plaintext password.
3. Digital Signatures
1. Prove that the message sent was not intercepted and changed.
2. We know that the digitial signature is not fake by ENCRYPTING the HASH of with the private key.  This is the digital signature.
1. The sig is then sent WITH the message.
2. The recipient then hashes the message for himself, and then DECRYPT the digital signature hash of with the public key and if the hashes match, we know that the message was not tampered with. hashing the private key.

Anyway, here's a table speaking to different types of hashs. One of the key things to know is that primitive hashes are vulnerable against collisions, and with that, become easier to crack. Several inputs can lead to the same output in the case of weak hashing algorithms.

MD5 (Message Digest v. 5)	Grandpa of Hashes (1992), 128 bit	128
SHA (Secure Hash Algorithm)	Family of Hashes developed by National Institute of Standards	168 Bit v 1. 512 has never had a collision though.
RIPEMD	128, 160, 256 and 360 versions

Go ahead and play with different hashes here. 

Or check the security of your password somewhere like here.

Webcrawling

Webcrawling.

 Sometimes you just need to grab a bunch of text from a website really quick and fast. This is where its nice to have handy a few web-crawling algorithms for some extreme copy and paste. I once had to quickly gather all the headlines from a website and there was just no time to copy and paste it all. So I imported BeautifulSoup and just skimmed for the headers of each application.

• Step 1) Import BeautifulSoup and the request packages. 
• Step 2) Make a request to your target website and safe the raw html document. 
• Step 3) Have your BeautifulSoup object skim through whatever it is you need to grab.

 Key Code to remember:  

from bs4 import BeautifulSoup 
import re '''This is the request package for making httprequests''' 
html_doc=""" """ 

def extractHeaders(string): 
    soup=BeautifulSoup(string, 'html.parser') 
        for tag in soup.find_all(re.compile("h4")): 
        print(tag)

def extractHeaders(string, tag): 
    soup=BeautifulSoup(string, 'html.parser')
        for tag in soup.find_all(re.compile(tag)): print(tag)

Key takeaway: Search by the html tag you want targeted, like 'h4' or 'title' or something.

I noticed that in this case however, its important to make your request <i>seem</i> like it is from an actual web browser. The headers parameter of the request can handle this. To be certain, set your headers as something that follows below.

headers={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:65.0) Gecko/20100101 Firefox/65.0'}

Then try running your request with that header as your header.

r= requests.get('target_html', headers=headers)