Category Archives: Coder

Usage of API documented as @since 1.8+

Usage of API documented as @since 1.8+

this may useful when development is performed under newer sdk version as the target platform for production

The configuration in pom.xml is.

        <plugin>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.8.0</version>
        </plugin>

Method 1: open project structure, select modules in the sidebar, and modify the language level in the sources window

Method 2: modify POM. XML to

        <plugin>
          <artifactId>maven-compiler-plugin</artifactId>
          <version>3.8.0</version>
          <configuration>
            <source>11</source>
            <target>11</target>
          </configuration>
        </plugin>

URL transcoding (including Chinese): 3A to ‘:’,% 2F to ‘/’

StringstrURL="";
try{
strURL=URLEncoder.encode(url,"utf-8");
}catch(UnsupportedEncodingExceptione){
//TODOAuto-generatedcatchblock
System.out.println("failed~");
e.printStackTrace();
}
strURL=strURL.replaceAll("%3A",":").replaceAll("%2F","/")
.replaceAll("%3F","?").replaceAll("%3D","=").replaceAll(
"%26","&");



:->3A-&>16*3+10-&>58-&>chr(58)=":"
/->2F-&>16*2+15-&>47-&>chr(47)="/"
-----------------------------------------------------
16*high bit + low bit
3A (hexadecimal) → 58 (decimal) → character (58) → display ":"
Encoding, not C++, this is to convert UTF8 to ANSI encoding.

When you use the mitmproxy tool under kalinux to view the request, you will find that the request is a symcd.com

I. When I used the mitmproxy tool under kalinux to check the requests, I found a problem

Under the request there is a request for ss.symcd.com What exactly is this for?

Morning! Hope you are having a great weekend. I’ve been experimenting with some network monitoring of HTTP requests and responses in Mozilla Firefox. While playing around with one of the tools I’m evaluating I noticed a request togv.symcd.com:

I had not heard of thesymcd.comdomain before so I got curious. The request is a “application/ocsp-request“.OCSPis a abbreviation forOnline Certificate Status Protocoland it is an Internet protocolused for retrievethe revocation status of a digital certificate.

That’s what the symcd.comconnection is about: Checking therevocation state for some certificate. The tool I used to track the network traffic does not have any advanced features to decode the OSCP communicationso I don’t know exactly what information Firefox requests from symcd.com.

So,who owns symcd.com?The WHOIS database answer isSymantec Corporation:

RegistrantOrganization:SymantecCorporation
RegistrantStreet:350EllisStreet
RegistrantCity:MountainView
RegistrantState/Province:CA
RegistrantPostalCode:94043
RegistrantCountry:US

Symcd.com was created on2013-12-12.

I did not find much information about gv.symdc.com, and the reason for that is probably because there’s a large number of subdomains used. I found thislist over at VirusTotal:

sm.symcd.com

gz.symcd.com

gp.symcd.com

tl.symcd.com

sn.symcd.com

tm.symcd.com

gq.symcd.com

sk.symcd.com

gw.symcd.com

si.symcd.com

gx.symcd.com

gk.symcd.com

s.symcd.com

sw.symcd.com

gu.symcd.com

sh.symcd.com

tf.symcd.com

t.symcd.com

tn.symcd.com

gv.symcd.com

ta.symcd.com

gd.symcd.com

st.symcd.com

tg.symcd.com

sr.symcd.com

sd.symcd.com

sf.symcd.com

sg.symcd.com

th.symcd.com

ga.symcd.com

gn.symcd.com

se.symcd.com

sv.symcd.com

tj.symcd.com

su.symcd.com

tb.symcd.com

ti.symcd.com

tc.symcd.com

sc.symcd.com

gm.symcd.com

sb.symcd.com

gb.symcd.com

ss.symcd.com

sj.symcd.com

gj.symcd.com

td.symcd.com

sa.symcd.com

tk.symcd.com

I checked a few of the domains, and they all resolved to the23.43.139.27 IP address.

Thanks for reading!

 

[LeetCode] 291. Word Pattern II

Given apatternand a stringstr, find ifstrfollows the same pattern.

Herefollowmeans a full match, such that there is a bijection between a letter inpatternand anon-emptysubstring instr.

Examples:

pattern ="abab", str ="redblueredblue"should return true.

pattern ="aaaa", str ="asdasdasdasd"should return true.

pattern ="aabb", str ="xyzabcxzyabc"should return false.

Notes:
You may assume bothpatternandstrcontains only lowercase letters.

290. Word pattern is an extension of word pattern. The difference is that there are no spaces in the word string, so we can’t split the words directly by spaces.

You can use the backtracking method to judge each case, use the hash table to establish the mapping between pattern characters and words, and use the variables p and R to record the position of the current recursive pattern characters and word strings. In the recursive function, if P and R are equal to the length of the pattern string and single word string respectively, it means that the matching is completed successfully, and return to ture. Otherwise, if a If the match is reached and the other one is not, it means that the match failed and returns false.

Solution: backtracking

Python:

# Time:  O(n * C(n - 1, c - 1)), n is length of str, c is unique count of pattern,
#                                there are H(n - c, c - 1) = C(n - 1, c - 1) possible splits of string,
#                                and each one costs O(n) to check if it matches the word pattern.
# Space: O(n + c)

class Solution(object):
    def wordPatternMatch(self, pattern, str):
        """
        :type pattern: str
        :type str: str
        :rtype: bool
        """
        w2p, p2w = {}, {}
        return self.match(pattern, str, 0, 0, w2p, p2w)


    def match(self, pattern, str, i, j, w2p, p2w):
        is_match = False
        if i == len(pattern) and j == len(str):
            is_match = True
        elif i < len(pattern) and j < len(str):
            p = pattern[i]
            if p in p2w:
                w = p2w[p]
                if w == str[j:j+len(w)]:  # Match pattern.
                    is_match = self.match(pattern, str, i + 1, j + len(w), w2p, p2w)
                # Else return false.
            else:
                for k in xrange(j, len(str)):  # Try any possible word
                    w = str[j:k+1]
                    if w not in w2p:
                        # Build mapping. Space: O(n + c)
                        w2p[w], p2w[p] = p, w;
                        is_match = self.match(pattern, str, i + 1, k + 1, w2p, p2w)
                        w2p.pop(w), p2w.pop(p);
                    if is_match:
                        break
        return is_match  

C++:

class Solution {
public:
    bool wordPatternMatch(string pattern, string str) {
        unordered_map<char, string> m;
        return helper(pattern, 0, str, 0, m);
    }
    bool helper(string pattern, int p, string str, int r, unordered_map<char, string&> &m) {
        if (p == pattern.size() && r == str.size()) return true;
        if (p == pattern.size() || r == str.size()) return false;
        char c = pattern[p];
        for (int i = r; i < str.size(); ++i) {
            string t = str.substr(r, i - r + 1);
            if (m.count(c) && m[c] == t) {
                if (helper(pattern, p + 1, str, i + 1, m)) return true;
            } else if (!m.count(c)) {
                bool b = false;
                for (auto it : m) {
                    if (it.second == t) b = true;
                } 
                if (!b) {
                    m[c] = t;
                    if (helper(pattern, p + 1, str, i + 1, m)) return true;
                    m.erase(c);
                }
            }
        }
        return false;
    }
};  

C++:

class Solution {
public:
    bool wordPatternMatch(string pattern, string str) {
        unordered_map<char, string> m;
        set<string> s;
        return helper(pattern, 0, str, 0, m, s);
    }
    bool helper(string pattern, int p, string str, int r, unordered_map<char, string> &m, set<string&> &s) {
        if (p == pattern.size() && r == str.size()) return true;
        if (p == pattern.size() || r == str.size()) return false;
        char c = pattern[p];
        for (int i = r; i < str.size(); ++i) {
            string t = str.substr(r, i - r + 1);
            if (m.count(c) && m[c] == t) {
                if (helper(pattern, p + 1, str, i + 1, m, s)) return true;
            } else if (!m.count(c)) {
                if (s.count(t)) continue;
                m[c] = t;
                s.insert(t);
                if (helper(pattern, p + 1, str, i + 1, m, s)) return true;
                m.erase(c);
                s.erase(t);
            }
        }
        return false;
    }
};

140. Word Break II

Title: 140. Word Break II

Given anon-emptystringsand a dictionarywordDictcontaining a list ofnon-emptywords, add spaces insto construct a sentence where each word is a valid dictionary word.Return all such possible sentences.

Note:

The same word in the dictionary may be reused multiple times in the segmentation.

You may assume the dictionary does not contain duplicate words.

Example 1:

Input:
s = "catsanddog" wordDict = ["cat", "cats", "and", "sand", "dog"] Output: [
 "cats and dog",
 "cat sand dog"
]

Example 2:

Input:
s = "pineapplepenapple"
wordDict = ["apple", "pen", "applepen", "pine", "pineapple"]
Output:
[
 "pine apple pen apple",
 "pineapple pen apple",
 "pine applepen apple"
]
Explanation: Note that you are allowed to reuse a dictionary word.

Example 3:

Input:
s = "catsandog"
wordDict = ["cats", "dog", "sand", "and", "cat"]
Output:
[]

Thinking of solving problems

In fact, the solution of this problem is the same as 139. Word break. You can refer to the idea of 139. Word break https://my.oschina.net/JiamingMai/blog/4331735 , slightly different from 139. Word break, this question needs to record all the cases in which h [0] [i] is true.

Time complexity analysis

In essence, it is the same as 139. Word break, so the time complexity is O (KN ^ 2), which is a reference for the analysis process https://my.oschina.net/JiamingMai/blog/4331735

Finally realized

java implementation

public class Solution {

    List<String&>[] records;

    private boolean[][] h;

    private boolean[][] r;

    public List<String&> wordBreak(String s, List<String&> wordDict) {
        int n = s.length();
        if (!fastWordBreakInternal(s, wordDict)) {
            return new ArrayList<&>();
        }
        boolean res = wordBreakInternal(s, wordDict);
        if (res) {
            return records[n];
        }
        return new ArrayList<&>();
    }

    public boolean fastWordBreakInternal(String s, List<String&> wordDict) {
        int n = s.length();
        h = new boolean[n+1][n+1];
        r = new boolean[n+1][n+1];
        initR(n, s, wordDict);
        h[0][0] = true;
        for (int i = 1; i <= n; i++) {
            for (int j = i; j &>= 0; j--) {
                if (!h[0][i]) {
                    h[0][i] = h[0][i-j] && r[i-j][i];
                } else {
                    break;
                }
            }
        }
        return h[0][n];
    }

    public boolean wordBreakInternal(String s, List<String&> wordDict) {
        int n = s.length();
        h = new boolean[n + 1][n + 1];
        r = new boolean[n + 1][n + 1];
        records = new List[n + 1];
        initR(n, s, wordDict);
        h[0][0] = true;
        for (int i = 1; i <= n; i++) {
            for (int j = i; j &>= 0; j--) {
                if (h[0][i - j] && r[i - j][i]) {
                    h[0][i] = true;
                    record(i, j, s);
                }
            }
        }
        return h[0][n];
    }

    private void record(int i, int j, String s) {
        if (null == records[i]) {
            records[i] = new ArrayList<&>();
        }
        String token = s.substring(i - j, i);
        if (null != records[i - j]) {
            for (String str : records[i - j]) {
                StringBuffer sb = new StringBuffer();
                sb.append(str).append(" ").append(token);
                records[i].add(sb.toString());
            }
        }  else {
            records[i].add(token);
        }
    }

    private void initR(int n, String s, List<String&> wordDict) {
        for (int i = 0; i <= n; i++) {
            for (int j = i; j <= n; j++) {
                r[i][j] = r(i, j, s, wordDict);
            }
        }
    }

    private boolean r(int i, int j, String s, List<String&> wordDict) {
        String token = s.substring(i, j);
        for (String word : wordDict) {
            if (word.equals(token)) {
                return true;
            }
        }
        return false;
    }

    public static void main(String[] args) {
        List<String&> wordDict = new ArrayList<&>();
        wordDict.add("a");
        wordDict.add("aa");
        wordDict.add("aaa");
        wordDict.add("aaaa");
        wordDict.add("aaaaa");
        wordDict.add("aaaaaa");
        wordDict.add("aaaaaaa");
        wordDict.add("aaaaaaaa");
        wordDict.add("aaaaaaaaa");
        wordDict.add("aaaaaaaaaa");
        String s = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaabaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa";
        Solution solution = new Solution();
        List<String&> res = solution.wordBreak(s, wordDict);
        res.stream().forEach(str -&> {System.out.println(str);});
    }

}

note that in the main method of the code, the input is a continuous string of a with only one B in the middle. In this case, if you record while calculating directly, it will time out. So here’s a trick to quickly determine whether the input string can be disassembled. If not, you can directly output an empty list. This example is just a case that cannot be disassembled.

The problem of WinDbg symbol

This article is excerpted from https://gclxry.com/problem-with-windbg-symbols/
Original author: gclxry
A

Recently, I analyzed dump to see the information in PEB, so I ran the! PEB command and returned as follows:

0:000> !peb
PEB at fffde000
*************************************************************************
***                                                                   ***
***                                                                   ***
***    Either you specified an unqualified symbol, or your debugger   ***
***    doesn't have full symbol information.  Unqualified symbol      ***
***    resolution is turned off by default. Please either specify a   ***
***    fully qualified symbol module!symbolname, or enable resolution ***
***    of unqualified symbols by typing ".symopt- 100". Note that     ***
***    enabling unqualified symbol resolution with network symbol     ***
***    server shares in the symbol path may cause the debugger to     ***
***    appear to hang for long periods of time when an incorrect      ***
***    symbol name is typed or the network symbol server is down.     ***
***                                                                   ***
***    For some commands to work properly, your symbol path           ***
***    must point to .pdb files that have full type information.      ***
***                                                                   ***
***    Certain .pdb files (such as the public OS symbols) do not      ***
***    contain the required information.  Contact the group that      ***
***    provided you with these symbols if you need this command to    ***
***    work.                                                          ***
***                                                                   ***
***    Type referenced: ntdll!_PEB                                    ***
***                                                                   ***
*************************************************************************
error 3 InitTypeRead( nt!_PEB at fffde000)...

Prompt me PDB symbol is wrong, cause! PEB can’t see valid information. I’m very strange, because I’ve been doing well before, and my debug symbol path hasn’t changed, as follows:

0:000&> .sympath
Symbol search path is: SRV*D:\symbols*https://msdl.microsoft.com/download/symbols;SRV*D:\symbols*http://172.xx.xx.xx/symbols/

Expanded Symbol search path is: srv*d:\symbols*https://msdl.microsoft.com/download/symbols;srv*d:\symbols*http://172.xx.xx.xx/symbols/

Confirm that the debugging symbol servers of Microsoft and our company are normal.

Then use! Symnoise to turn on the symbol load noise mode,. Reload/F ntdll.dll See how to parse the load symbol.

0:000> !sym noisy
noisy mode - symbol prompts on
0:000> .reload /f ntdll.dll
SYMSRV:  BYINDEX: 0x9
         d:\symbols*https://msdl.microsoft.com/download/symbols
         ntdll.dll
         589C957A180000
SYMSRV:  UNC: d:\symbols\ntdll.dll\589C957A180000\ntdll.dll - path not found
SYMSRV:  UNC: d:\symbols\ntdll.dll\589C957A180000\ntdll.dl_ - path not found
SYMSRV:  UNC: d:\symbols\ntdll.dll\589C957A180000\file.ptr - path not found
SYMSRV:  HTTPGET: /download/symbols/ntdll.dll/589C957A180000/ntdll.dll
SYMSRV:  HttpSendRequest: 800C2EFD - ERROR_INTERNET_CANNOT_CONNECT
SYMSRV:  RESULT: 0x800C2EFD
SYMSRV:  BYINDEX: 0xA
         d:\symbols*http://172.xx.xx.xx/symbols/
         ntdll.dll
         589C957A180000
SYMSRV:  UNC: d:\symbols\ntdll.dll\589C957A180000\ntdll.dll - path not found
SYMSRV:  UNC: d:\symbols\ntdll.dll\589C957A180000\ntdll.dl_ - path not found
SYMSRV:  UNC: d:\symbols\ntdll.dll\589C957A180000\file.ptr - path not found
SYMSRV:  HTTPGET: /symbols//ntdll.dll/589C957A180000/ntdll.dll
SYMSRV:  HttpQueryInfo(HTTP_QUERY_CONTENT_LENGTH): 800C2F76 - ERROR_HTTP_HEADER_NOT_FOUND
SYMSRV:  HttpQueryInfo: 80190194 - HTTP_STATUS_NOT_FOUND
SYMSRV:  HTTPGET: /symbols//ntdll.dll/589C957A180000/ntdll.dl_
SYMSRV:  HttpQueryInfo(HTTP_QUERY_CONTENT_LENGTH): 800C2F76 - ERROR_HTTP_HEADER_NOT_FOUND
SYMSRV:  HttpQueryInfo: 80190194 - HTTP_STATUS_NOT_FOUND
SYMSRV:  HTTPGET: /symbols//ntdll.dll/589C957A180000/file.ptr
SYMSRV:  HttpQueryInfo(HTTP_QUERY_CONTENT_LENGTH): 800C2F76 - ERROR_HTTP_HEADER_NOT_FOUND
SYMSRV:  HttpQueryInfo: 80190194 - HTTP_STATUS_NOT_FOUND
SYMSRV:  RESULT: 0x80190194
DBGHELP: C:\Program Files (x86)\Windows Kits\10\Debuggers\ntdll.dll - file not found
DBGENG:  C:\Windows\SysWOW64\ntdll.dll image header does not match memory image header.
DBGENG:  C:\Windows\SysWOW64\ntdll.dll - Couldn't map image from disk.
Unable to load image C:\Windows\SysWOW64\ntdll.dll, Win32 error 0n2
DBGENG:  ntdll.dll - Partial symbol image load missing image info
DBGHELP: Module is not fully loaded into memory.
DBGHELP: Searching for symbols using debugger-provided data.
SYMSRV:  BYINDEX: 0xB
         d:\symbols*https://msdl.microsoft.com/download/symbols
         wntdll.pdb
         611AE48A538F4C0B82726D75DE80A6A92
SYMSRV:  UNC: d:\symbols\wntdll.pdb\611AE48A538F4C0B82726D75DE80A6A92\wntdll.pdb - path not found
SYMSRV:  UNC: d:\symbols\wntdll.pdb\611AE48A538F4C0B82726D75DE80A6A92\wntdll.pd_ - path not found
SYMSRV:  UNC: d:\symbols\wntdll.pdb\611AE48A538F4C0B82726D75DE80A6A92\file.ptr - path not found
SYMSRV:  HTTPGET: /download/symbols/wntdll.pdb/611AE48A538F4C0B82726D75DE80A6A92/wntdll.pdb
SYMSRV:  HttpSendRequest: 800C2EFD - ERROR_INTERNET_CANNOT_CONNECT
SYMSRV:  RESULT: 0x800C2EFD
SYMSRV:  BYINDEX: 0xC
         d:\symbols*http://172.xx.xx.xx/symbols/
         wntdll.pdb
         611AE48A538F4C0B82726D75DE80A6A92
SYMSRV:  UNC: d:\symbols\wntdll.pdb\611AE48A538F4C0B82726D75DE80A6A92\wntdll.pdb - path not found
SYMSRV:  UNC: d:\symbols\wntdll.pdb\611AE48A538F4C0B82726D75DE80A6A92\wntdll.pd_ - path not found
SYMSRV:  UNC: d:\symbols\wntdll.pdb\611AE48A538F4C0B82726D75DE80A6A92\file.ptr - path not found
SYMSRV:  HTTPGET: /symbols//wntdll.pdb/611AE48A538F4C0B82726D75DE80A6A92/wntdll.pdb
SYMSRV:  HttpQueryInfo(HTTP_QUERY_CONTENT_LENGTH): 800C2F76 - ERROR_HTTP_HEADER_NOT_FOUND
SYMSRV:  HttpQueryInfo: 80190194 - HTTP_STATUS_NOT_FOUND
SYMSRV:  HTTPGET: /symbols//wntdll.pdb/611AE48A538F4C0B82726D75DE80A6A92/wntdll.pd_
SYMSRV:  HttpQueryInfo(HTTP_QUERY_CONTENT_LENGTH): 800C2F76 - ERROR_HTTP_HEADER_NOT_FOUND
SYMSRV:  HttpQueryInfo: 80190194 - HTTP_STATUS_NOT_FOUND
SYMSRV:  HTTPGET: /symbols//wntdll.pdb/611AE48A538F4C0B82726D75DE80A6A92/file.ptr
SYMSRV:  HttpQueryInfo(HTTP_QUERY_CONTENT_LENGTH): 800C2F76 - ERROR_HTTP_HEADER_NOT_FOUND
SYMSRV:  HttpQueryInfo: 80190194 - HTTP_STATUS_NOT_FOUND
SYMSRV:  RESULT: 0x80190194
DBGHELP: wntdll.pdb - file not found
*** WARNING: Unable to verify timestamp for ntdll.dll
*** ERROR: Module load completed but symbols could not be loaded for ntdll.dll
DBGHELP: ntdll - no symbols loaded

************* Symbol Loading Error Summary **************
Module name            Error
ntdll                  The system cannot find the file specified
                The SYMSRV client failed to find a file in the UNC store, or there
                is an invalid UNC store (an invalid path or the pingme.txt file is
                not present in the root directory), or the file is present in the
                symbol server exclusion list.

Found the output SYMSRV: HttpSendRequest: 800C2EFD – ERROR_INTERNET_CANNOT_CONNECT, could it be that Microsoft’s symbolic server is walled? Manually open the ntdll.dll symbolic url in the browser https://msdl.microsoft.com/download/symbols/ntdll.dll/589C957A180000/ntdll.dll and found a redirect to https://vsblobprodscussu5shard87.blob.core.windows.net/b-4712e0edc5a240eabf23330d7df68e77/787A6C8378595D38A99B4DAFBE7316691BFBE38E4D0CA1A7637EE21A8140836900.blob?sv=2017-04-17&sr=b&si=1&sig=SX5nGwAekvPaY8jUMSUlZRHUcLEH6rZ6A8Y39HjQwfM%3D&spr=https&se=2020-01-07T07%3A28%3A15Z&rscl=x-e2eid-12793ff4-22fd46ce-b05476a0-93cdf775-session-ca085441-b4d94ca1-a365b6a1-3c06aa71 。

This url can really only be opened with a proxy.

How VC6 makes dialog respond to WM_ Char message

How VC6 makes dialog respond to WM_ Char message

The solution is to override the function PreTranslateMessage, which does the processing and sets the focus on the main window.
The specific code is as follows.

BOOLCMfcDlgMainDlg::PreTranslateMessage(MSG*pMsg)
{
if(WM_KEYDOWN==pMsg->message||WM_CHAR==pMsg->message)
{
pMsg->hwnd=m_hWnd;
returnFALSE;
}
returnCDialog::PreTranslateMessage(pMsg);
}

 

TCP segment of a reassembled PDU

Today, we use the windows search function to search the contents of a shared folder on the network. We find that the traffic is huge when we search the network files. Curious to use Wireshark to capture packets, I found that there are many prompt messages of “TCP segment of a reassembled PDU” in Wireshark info column. Don’t understand Baidu for a while, found that everyone is asking this question, online and no good answer. Thinking that “TCP segment of a reassembled PDU” is just a Wireshark prompt, what kind of prompt will be given in Sniffer Pro?Open the same trace with sniffer and find that it prompts “continuation of missing frame” and “continuation of frame XX”. Now you probably know “TCP segment of a reassembled PDU” What does “PDU” mean?In fact, when the host responds to a query or command, if it needs to respond to a lot of data (information) and the data exceeds the maximum MSS of TCP, the host will send multiple packets to transmit the data (Note: these packets are not fragmented). For Wireshark, these packets corresponding to the same query command are marked with “TCP segment of a reassembled PDU”

How does Wireshark identify that multiple packets are responses to the same query packet?Wireshark identifies these packets according to the sequence number. The ACK number of these packets is the same. Of course, the number value is the same as the next sequence number in the query packet.

[background knowledge] MTU: maximum transmission unit

MSS: maximum segment size

PPPoE: PPP over Ethernet

[analysis process] let’s talk about the maximum transmission unit of MTU first. This maximum transmission unit is actually closely related to the link layer protocol. Let’s carefully recall the structure of Ethernet II frame DMAC + SMAC + type + data + CRC. Due to the electrical limitation of Ethernet transmission, each Ethernet frame has a minimum size of 64bytes, which can’t be exceeded 1518 bytes. For Ethernet frames less than or greater than this limit, we can regard them as wrong data frames. General Ethernet forwarding devices will discard these data frames. (Note: data frames smaller than 64bytes are generally caused by “fragmentation” caused by Ethernet conflict, line interference or bad Ethernet interface. For data frames larger than 1518bytes, we generally call them giant frames, which are generally caused by line interference or bad Ethernet interface.)

Since the largest data frame of Ethernet II is 1518bytes, the header of Ethernet frame (DMAC destination MAC address 48bit = 6bytes + SMAC source MAC address 48bit = 6bytes + 2bytes in type field) 14bytes and the CRC check part 4bytes at the end of frame (sometimes called FCS by this department) are removed, so the place where the upper layer protocol is carried is the most important part of data field We call it MTU when the size is only 1500 bytes. This is what the network layer protocol is very concerned about, because the network layer protocol, such as IP protocol, will decide whether to fragment the data passed down from the upper layer according to this value. It’s just like a box can’t hold a big piece of bread. We need to slice the bread and put it in multiple boxes.

When two remote PCs are interconnected, their data need to go through a lot of routers and a variety of network media to reach the opposite end. The MTU of different media in the network is different, just like a long section of water pipe, which is composed of water pipes of different thickness (different MTU). The maximum water flow through this section of water pipe is determined by the thinnest water pipe in the middle.

For the upper layer protocols of the network layer (let’s take TCP/IP protocol family as an example), they don’t care about the thickness of the water pipe. They think it’s a matter of the network layer. Network layer IP protocol will check the size of each packet from the upper layer protocol, and decide whether to “slice” according to the size of the local MTU. The biggest disadvantage of fragmentation is that it reduces the transmission performance. What can be done at one time can be done many times, so we often pay attention to this in the implementation of the higher layer of the network layer (that is, the transmission layer)! For some reasons, some high-level officials will ask me that this bread can’t be sliced, I want to complete the bread, so I will add a label in the IP packet header: DF (donot fragment). In this way, when the IP packet is transmitted in a large network (water pipe), if the MTU is smaller than the IP packet, the forwarding device will discard the packet according to the requirements. Then an error message is returned to the sender. This often causes some communication problems, but fortunately, most of the network links are mtu1500 or larger.

As far as UDP is concerned, it is a connectionless protocol, and it doesn’t care much about the arrival order and correct arrival of packets. Therefore, there are no special requirements for fragmentation in general UDP applications.

For TCP protocol, it is not the same. This protocol is connection oriented. For TCP protocol, it is very concerned about the arrival order of packets and whether there are errors in transmission. Therefore, some TCP applications have requirements for fragmentation – no fragmentation (DF).

Please come out today’s third pig foot: MSS. The abbreviation of MSS maximum transmission size is a concept in TCP protocol. MSS is the largest data segment that TCP packets can transmit each time. In order to achieve the best transmission efficiency, the MSS value of both sides is usually negotiated when TCP protocol establishes a connection. When TCP protocol implements this value, it is often replaced by MTU value (20bytes of IP packet header and 20bytes of TCP packet header need to be subtracted), so the MSS value is usually 1460. Both sides of the communication will determine the maximum MSS value of this connection according to the minimum MSS value provided by both sides.

After introducing the three pigtails, let’s go back to the problem in the foreword. Let’s imagine that if we adjust the maximum MSS of each TCP connection on the intermediate router, so that the maximum MSS value of the PPPoE link plus the packet header and packet tail will not exceed the MTU size of the PPPoE This will not cause the problem of communication failure. Therefore, the above problem can be solved by iptcp adjust MSS 1452.

Of course, the problem can also be solved by modifying the MTU of PC.

Wireshark window size value and calculated window size

Recently in the project, I found that when I contracted out two machines, one of them was slow to collect. I wanted to take a look at the package. Before, I just looked at the package.

This time I really care about the window. At first glance, I think it’s strange. I used the following command to catch the bag:

tcpdump-ieth0tcpandport9981-w/data/steven/tcpdump/tcpdump.cap-C1-s0&

Mainly is – C, limits the file size to 1m to cut the file. Because the two programs are connected for a long time, there is no syn handshake process for the packets caught in the middle.

Looking at the data, I think it’s very strange.

99 machine told 100 machine, win = 36, but then 100 sent a len = 180 data to 99, which is not in line with the implementation of sliding window!

For a long time, I didn’t find the relevant information, because the method I used to check was wrong. Later, I asked my colleagues to restart the program and catch another bag with a handshake

Syn package, a look, it seems to be more normal:

This package win looks a bit normal, although there is still a problem, 99 win is much smaller than 100. Why do I think the first bag is abnormal

Because RFC has said that MSS Ethernet is 1460 bytes (1500 – 40), and the minimum MSS seems to be 536 bytes (576, the minimum number of bytes of reorganization buffer – 40).

Click on the syn package, and the following two fields attract my attention:

Google took a look at Wireshark window size value calculated window size and found two articles in the reference link

You can only look at the first one when you are surrounded by a wall

Since the TCP header window field is only 16bit, which represents 64K at most, an optional magnification is used to represent a larger window.

1. When TCP shakes hands three times, in syn or syn, ACK packet, inform options optional information, inform the other party that the amplification factor will be used.

2. Syn itself is not amplified

3. Therefore, the window size value represents the value of the message, and the calculated window size represents the enlarged value, that is, the actual available value

This value should be calculated by Wireshark for the sake of friendliness.

 

Back to the question I said at the beginning, I used the – C parameter when I tcpdump, that is, when the file exceeds 1m, it will be cut into multiple files, when your

When the file does not contain the three times handshake packet (SYN), Wireshark will not know your magnification, so the displayed value will have my question:

Why is win only 36 but can be len = 180.

As you can see, in the tcpdump file without syn package, Wireshark does not know your multiple, so – 1 (unknown) is displayed, and

The value win = 37 is displayed outside your package. If you calculate 128 to correct it (actually 128), the calculated window size is correct

It should show 37 * 128 = 4736, which is not much different from the following. It’s much bigger than 180. Of course, it can send 180.

The following package is caught in the included syn. He can tell you that the multiple is 128, which is actually 5760

Finally, there are two questions about this matter:

Why are the windows on both sides so different when shaking hands?100 is at least 2.4 times of 99.

when TCP starts this magnification.

A: it seems that Linux has a kernel configuration parameter

a[ steven@KFJK ~]$ more /proc/sys/net/ipv4/tcp_ window_ Scaling

1

here is a brief description of [original] adjustment algorithm of TCP receiving window (Part 1)

“if the maximum value of receiving window is limited by TCP_ Rmem [2] = 4194304, then RCV_ Wscale = 7, and the window enlargement is 128. ”

and the setting of my system seems to be just right (but the factor is far more than that):

[ steven@KFJK ~]$ cat /proc/sys/net/ipv4/tcp_ Rmem

4096 87380 4194304

the article recommended by colleagues is good, TCP windows and window scaling

by the way, the article recommended by colleagues Wireshark TCP protocol analysis is also very good.

If you have time, you’d better read the detailed explanation of TCP/IP. Here are two articles written by an expert, fast food:

Things about TCP in coolshell (Part 1) and things about TCP (Part 2)

 

You have not completed the merge: You have not concluded your merge (MERGE_HEAD exists)

Problem:

I made a branch called ‘f’ and did a checkout to master. When I tried the git pull command I got this message:

You have not concluded your merge (MERGE_HEAD exists).
Please, commit your changes before you can merge.

When I try the git status , it gave me the following:

On branch master
# Your branch and 'origin/master' have diverged,
# and have 1 and 13 different commit(s) each, respectively.
#
# Changes to be committed:
#
#   modified:   app/assets/images/backward.png
#   modified:   app/assets/images/forward.png
#   new file:   app/assets/images/index_background.jpg
#   new file:   app/assets/images/loading.gif
#   modified:   app/assets/images/pause.png
#   modified:   app/assets/images/play.png
#   new file:   app/assets/javascripts/jquery-ui-bootstrap.js
#   new file:   app/assets/stylesheets/jquery-ui-bootstrap.css
#   modified:   app/controllers/friends_controller.rb
#   modified:   app/controllers/plays_controller.rb
#   modified:   app/mailers/invite_friends_mailer.rb
#   modified:   app/mailers/send_plays_mailer.rb
#   modified:   app/mailers/shot_chart_mailer.rb
#   modified:   app/views/friends/show_plays.html.erb
#   modified:   app/views/layouts/application.html.erb
#   modified:   app/views/plays/_inbox_table.html.erb
#   modified:   app/views/plays/show.html.erb
#   modified:   app/views/welcome/contact_form.html.erb
#   modified:   app/views/welcome/index.html.erb
#   modified:   log/development.log
#   modified:   log/restclient.log
#   new file:   tmp/cache/assets/C1A/C00/sprockets%2Fb7901e0813446f810e560158a1a97066
#   modified:   tmp/cache/assets/C64/930/sprockets%2F65aa1510292214f4fd1342280d521e4c
#   new file:   tmp/cache/assets/C73/C40/sprockets%2F96912377b93498914dd04bc69fa98585
#   new file:   tmp/cache/assets/CA9/090/sprockets%2Fa71992733a432421e67e03ff1bd441d8
#   new file:   tmp/cache/assets/CCD/7E0/sprockets%2F47125c2ebd0e8b29b6511b7b961152a1
#   modified:   tmp/cache/assets/CD5/DD0/sprockets%2F59d317902de6e0f68689899259caff26
#   modified:   tmp/cache/assets/CE3/080/sprockets%2F5c3b516e854760f14eda2395c4ff2581
#   new file:   tmp/cache/assets/CED/B20/sprockets%2F423772fde44ab6f6f861639ee71444c4
#   new file:   tmp/cache/assets/D0C/E10/sprockets%2F8d1f4b30c6be13017565fe1b697156ce
#   new file:   tmp/cache/assets/D12/290/sprockets%2F93ae21f3cdd5e24444ae4651913fd875
#   new file:   tmp/cache/assets/D13/FC0/sprockets%2F57aad34b9d3c9e225205237dac9b1999
#   new file:   tmp/cache/assets/D1D/DE0/sprockets%2F5840ff4283f6545f472be8e10ce67bb8
#   new file:   tmp/cache/assets/D23/BD0/sprockets%2F439d5dedcc8c54560881edb9f0456819
#   new file:   tmp/cache/assets/D24/570/sprockets%2Fb449db428fc674796e18b7a419924afe
#   new file:   tmp/cache/assets/D28/480/sprockets%2F9aeec798a04544e478806ffe57e66a51
#   new file:   tmp/cache/assets/D3A/ED0/sprockets%2Fcd959cbf710b366c145747eb3c062bb4
#   new file:   tmp/cache/assets/D3C/060/sprockets%2F363ac7c9208d3bb5d7047f11c159d7ce
#   new file:   tmp/cache/assets/D48/D00/sprockets%2Fe23c97b8996e7b5567a3080c285aaccb
#   new file:   tmp/cache/assets/D6A/900/sprockets%2Fa5cece9476b21aa4d5f46911ca96c450
#   new file:   tmp/cache/assets/D6C/510/sprockets%2Fb086a020de3c258cb1c67dfc9c67d546
#   new file:   tmp/cache/assets/D70/F30/sprockets%2Facf9a6348722adf1ee7abbb695603078
#   new file:   tmp/cache/assets/DA3/4A0/sprockets%2F69c26d0a9ca8ce383e20897cefe05aa4
#   new file:   tmp/cache/assets/DA7/2F0/sprockets%2F61da396fb86c5ecd844a2d83ac759b4b
#   new file:   tmp/cache/assets/DB9/C80/sprockets%2F876fbfb9685b2b8ea476fa3c67ae498b
#   new file:   tmp/cache/assets/DBD/7A0/sprockets%2F3640ea84a1dfaf6f91a01d1d6fbe223d
#   new file:   tmp/cache/assets/DC1/8D0/sprockets%2Fe5ee1f1cfba2144ec00b1dcd6773e691
#   new file:   tmp/cache/assets/DCC/E60/sprockets%2Fd6a95f601456c93ff9a1bb70dea3dfc0
#   new file:   tmp/cache/assets/DF1/130/sprockets%2Fcda4825bb42c91e2d1f1ea7b2b958bda
#   new file:   tmp/cache/assets/E23/DE0/sprockets%2Fb1acc25c28cd1fabafbec99d169163d3
#   new file:   tmp/cache/assets/E23/FD0/sprockets%2Fea3dbcd1f341008ef8be67b1ccc5a9c5
#   modified:   tmp/cache/assets/E4E/AD0/sprockets%2Fb930f45cfe7c6a8d0efcada3013cc4bc
#   new file:   tmp/cache/assets/E63/7D0/sprockets%2F77de495a665c3ebcb47befecd07baae6
#   modified:   tmp/pids/server.pid
#
# Untracked files:
#   (use "git add <file&>..." to include in what will be committed)
#
#   Coachbase/
#   log/development.log.orig
#   log/restclient.log.orig

What should I do?